Systems and methods for a supplemental display screen

ABSTRACT

Systems and methods are disclosed for using a supplementary display device in conjunction with a first device, where the first device may have a smaller display than the supplementary display device. Icon images corresponding to applications installed on the first device are accessed and wirelessly transmitted to the supplementary display device. Touch data, corresponding to a user touch input on a touchscreen of the supplementary display device, is received at the first device. The touch data includes coordinate data corresponding to the user touch input on a touchscreen of the supplementary display device. Based at least in part on the touch data, a determination is to be made as whether an application on the first device is to be launched. The application is launched on the first device and a user interface corresponding to the launched application is transmitted to the supplementary display device.

INCORPORATION BY REFERENCE TO ANY PRIORITY APPLICATIONS

Any and all applications for which a foreign or domestic priority claimis identified in the Application Data Sheet as filed with the presentapplication are hereby incorporated by reference under 1 CFR 1.57.

BACKGROUND

The present invention is generally related to computing devices havingwireless interfaces and displays.

DESCRIPTION OF THE RELATED ART

Many portable personal electronic devices utilize screen sizes of lessthan six inches to enable them to be more portable by fitting intopockets or small bags. While this small size provides portableconvenience, the smaller screen size relative to desktop computers orportable tablet computers makes it more difficult for users to watchvideos, read e-mail, type, and interact with the device.

SUMMARY

An aspect of the disclosure relates to electronic display replicator fora primary portable computing device may comprise a case, an electronicdisplay screen coupled to the case, and a control system disposed withinthe case. The control system is configured to establish a wirelesscommunication channel between the electronic display replicator and theprimary portable computing device to allow the transposition of datafrom a display screen of the primary portable computing device onto theelectronic display screen. The control system optionally allows theelectronic display screen of the electronic display replicator to act inplace of the first display to operate and control the primary portablecomputing device.

An aspect of the disclosure relates to a supplemental display deviceconfigured to be used with a first computing device having a firstdisplay, comprising: a casing; a touchscreen display; a control systemdisposed within the casing and electrically coupled to the touchscreendisplay, wherein the control system is configured to perform operationscomprising: establish a direct wireless connection with the firstcomputing device; access, using the direct wireless connection,application icons from the first computing device, the application iconscorresponding to applications installed on the first computing device;display the application icons from the first computing device in a gridformat on the touchscreen display; detect an orientation of thesupplemental display device; orient the application icons on thetouchscreen display in accordance with the detected orientation of thesupplemental display device and independent of an orientation of thefirst computing device; capture a user input on the touchscreen display;transmit the captured user input to the first computing device, whereinthe captured user input transmitted to the first computing device isconfigured to cause a corresponding action to be performed on the firstcomputing device; and wirelessly receive content transmitted from thefirst computing device, wherein the content is transmitted from thefirst computing device at least partly in response to the transmittedcaptured user input.

An aspect of the disclosure relates to a computer implementedcomprising: establishing, by a supplemental display device having atouchscreen display, a wireless connection with a first computing devicehaving a first display; accessing, by the supplemental display device, afirst plurality of application icons over the wireless connection fromthe first computing device, the first plurality application iconscorresponding to applications installed on the first computing device;displaying, by the supplemental display device, the first plurality ofapplication icons from the first computing device in a grid format onthe touchscreen display of the supplemental display device; capturing auser touch on the touchscreen display of the supplemental displaydevice; transmitting, by the supplemental display device, the captureduser touch to the first computing device, wherein the captured usertouch transmitted to the first computing device is configured to cause acorresponding action to be performed on the first computing device; andwirelessly receiving, by the supplemental display device, contenttransmitted from the first computing device, wherein the content istransmitted from the first computing device at least partly in responseto the transmitted captured user touch.

An aspect of the disclosure relates to a computer-implemented methodcomprising: determining by a first device configuration data for thefirst device, wherein the first device hosts a communication andsynchronization application; wirelessly transmitting the determinedfirst device configuration data from the first device to a supplementarydisplay device; determining, by the first device, a communicationenvironment of the first device and generating correspondingcommunication environment data, the communication environment dataindicating whether the first device is wirelessly connected to a WIFIaccess point and whether the first device has access to a cellularnetwork; wirelessly transmitting the first device communicationenvironment data from the first device to the supplementary displaydevice, the communication environment data indicating whether the firstdevice is wirelessly connected to a WIFI access point and whether thefirst device has access to a cellular network; receiving from thesupplementary display device by the first device touch datacorresponding to a user touch input on a touchscreen of thesupplementary display device, the touch data including at leastcoordinate data corresponding to the user touch input; determining bythe first device an action to be taken in response to the touch datareceived from the supplementary display device; taking, by the firstdevice, the determined action; and wirelessly transmitting contentassociated with the taken action from the first device to thesupplementary display device.

An aspect of the disclosure relates to a computer system, comprising: atouchscreen display; computing hardware; a first wireless networkinginterface; a second wireless networking interface; a third wirelessnetworking interface; non-transitory memory that stores instructionsthat when executed by the computing hardware cause the computer systemto perform operations comprising: accessing configuration data of thecomputer system; wireles sly transmitting the configuration data to asupplementary display device using the first wireless networkinginterface or the second wireless networking interface; determining acommunication environment of the computer system and generatingcorresponding communication environment data, the communicationenvironment data indicating whether the computer system is wirelesslyconnected to a network access point using the second wireless networkinginterface and indicating whether the computer system has access to acellular network via the third wireless networking interface; wirelesslytransmitting the communication environment data to the supplementarydisplay device; receiving from the supplementary display device touchdata corresponding to a user touch input on a touchscreen of thesupplementary display device, the touch data including at least touchcoordinate data; determining at the computer system an action to betaken in response to the touch data, including at least the touchcoordinate data received from the supplementary display device; andtaking, by the computer system, the determined action.

An aspect of the disclosure relates to a non-transitory memory thatstores instructions that when executed by a computing system cause thecomputer system to perform operations comprising: accessingconfiguration data of the computer system; wirelessly transmitting theconfiguration data to a supplementary display device using a firstwireless networking interface or a second wireless networking interface;determining a communication environment of the computer system andgenerating corresponding communication environment data, thecommunication environment data indicating whether the computer system iswirelessly connected to a network access point using the second wirelessnetworking interface and indicating whether the computer system hasaccess to a cellular network via a third wireless networking interface;wirelessly transmitting the communication environment data to thesupplementary display device; receiving from the supplementary displaydevice touch data corresponding to a user touch input on a touchscreenof the supplementary display device, the touch data including at leasttouch coordinate data; determining an action to be taken in response tothe touch data, including at least the touch coordinate data, receivedfrom the supplementary display device; and taking the determined action.

An aspect of the disclosure relates to a non-transitory memory thatstores instructions that when executed by a computing system cause thecomputer system to perform operations comprising: identifying one ormore applications installed on the computing system; accessing iconimages corresponding to the one or more applications installed on thecomputing system; wirelessly transmitting the icon images correspondingto the one or more applications installed on the computing system to asupplementary display device; receiving from the supplementary displaydevice touch data corresponding to a user touch input on a touchscreenof the supplementary display device, the touch data including at leasttouch coordinate data; determining based at least in part on the touchdata, including at least the coordinate data, whether an application onthe computer system is to be launched; in response to determining thatan application on the computer system is to be launched, launching theapplication; and transmitting a user interface corresponding to thelaunched application to the supplementary display device.

An aspect of the disclosure relates to a method, comprising: identifyingone or more applications installed on a first computer system;accessing, by the first computer system, icon images corresponding tothe one or more applications installed on the first computer system;wirelessly transmitting, by the first computer system, the icon imagescorresponding to the one or more applications installed on the firstcomputer system to a supplementary display device; receiving, at thefirst computer system from the supplementary display device, touch datacorresponding to a user touch input on a touchscreen of thesupplementary display device; determining, by the first computer system,based at least in part on the touch data, whether an application on thefirst computer system is to be launched; in response to determining thatan application on the first computer system is to be launched, launchingthe application; and transmitting data corresponding to the launchedapplication to the supplementary display device.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will now be described with reference to the drawingssummarized below. Throughout the drawings, reference numbers may bere-used to indicate correspondence between referenced elements. Thedrawings are provided to illustrate example embodiments described hereinand are not intended to limit the scope of the disclosure.

FIG. 1 representatively illustrates an example electronic displayreplicator according to one embodiment of the technology;

FIG. 2 representatively illustrates the various components of theexample electronic display replicator;

FIG. 3 representatively illustrates the example electronic displayreplicator communicatively coupled to a primary portable computingdevice;

FIG. 4 representatively illustrates an example method of establishing acommunication channel between the electronic display replicator andanother electronic device;

FIG. 5 illustrates an example operational architecture;

FIG. 6 illustrates example communication and synchronizationarchitectures;

FIG. 7 illustrates an example device registration process;

FIG. 8 illustrates an example replication process;

FIG. 9 illustrates an example distributed content access process;

FIG. 10 illustrates an example communication management process.

FIG. 11 illustrates an example process of adapting a supplementaldisplay for use with other devices.

DESCRIPTION

The present technology may be described in terms of functional blockcomponents and various processing steps. Such functional blocks may berealized by any number of components configured to perform the specifiedfunctions and achieve the various results. For example, the presenttechnology may employ various types of portable computing devices,display systems, communication protocols, networks, software/firmware,and the like. In addition, the present technology may be practiced inconjunction with any number of electronic devices and communicationnetworks, and the system described is merely one exemplary applicationfor the technology.

Systems and methods for an electronic display replicator according tovarious aspects of the present technology may operate in conjunctionwith any suitable portable electronic device and communication network.Various representative implementations of the present technology may beapplied to any system for communicating information/data between twoelectronic devices.

Referring now to FIGS. 1-3, an example electronic display replicator 100may be configured to operate in conjunction with a primary portablecomputing device 301. The example electronic display replicator 100 maybe configured to receive information and/or data from the primaryportable computing device 301 via a wireless communication channel 302established between the electronic display replicator 100 and theprimary portable computing device 301. The electronic display replicator100 may further be configured to operate the primary portable computingdevice 301 based on input commands received from a user.

Referring now to FIGS. 1 and 2, optionally the electronic displayreplicator 100 may comprise a casing 101 configured to hold a displayscreen 102. The casing 101 may comprise any suitable device or systemfor housing, holding, or otherwise enclosing various components. Forexample, the casing 101 may comprise an upper surface (not shown) and alower surface (not shown). The upper and lower surface may be coupledtogether to form an interior volume suitably configured to house aplurality of electronic components and the display screen 102.

The casing 101 may comprise any material such as plastics, metals,composite, and/or the like such as: aluminum; molded polymers;polymer-based composites; fiber-reinforced polymers; injection moldedpolycarbonate; acrylonitrile butadiene styrene (“ABS”) plastic;polypropylene; polyethylene; and polystyrene; polyvinyl chloride(“PVC”); or any suitable combination or mixture thereof. The casing 101may be manufactured as a single piece, or the casing 101 may bemanufactured in separate pieces and then later assembled into a singlepiece.

The casing 101 may comprise one or more functional control options 103,104, 105 to allow a user to control or operate various aspects of theelectronic display replicator 100. The control options 103, 104, 105 maycomprise any suitable system or device configured to operate theelectronic display replicator 100, control functions, and/or its variouscomponents. The control options 103, 104, 105 may be electrically and/orcommunicatively coupled to the various electrical components of theelectronic display replicator 100. The control options 103, 104, 105 maycomprise any suitable button, dial, slider, switch, tab such as: volumecontrols; on/off controls; a charging port; audio port; and/or the like.The control options 103, 104, 105 may be configured to be affixed to thecasing 101, detachably coupled to the casing 101, integrated with orinto the casing 101, and/or otherwise positioned along an outer surfaceof the casing 101.

For example, the control option for a power switch 103 may beelectrically and/or communicatively coupled to a component of theelectronic display replicator 100 configured to turn the electronicdisplay replicator 100 on and off. In another example, a charging portcontrol option 105 may be electrically and/or communicatively coupled toa power source (battery) 107 such that the charging port control option105 is configured to provide electrical power to the power source 107 tofacilitate charging.

Optionally, the electronic display replicator may comprise sound system106. The sound system 106 may comprise any suitable system or deviceconfigured to emit an audible sound. For example, the sound system 106may comprise a set of speakers configured to emit audible informationregarding either one of the electronic display replicator 100, theprimary portable computing device 201, and/or a combination of the twovia the wireless communication channel 202.

Optionally, some control options may be displayed on the display screenrather than embodied as a physical control button or switch. In such aconfiguration, one or more control options may be provided to the uservia the electronic display screen 102 once the electronic displayreplicator 100 has been turned on and/or otherwise activated. Forexample, the control options may appear on the electronic display screen102 such that the user can use their finger to activate the variouscontrol options for volume, brightness, and/or the like.

Optionally, the electronic display replicator 100 may also be configuredto be powered on without the need for a physical toggle and/or an on/offswitch. For example, the electronic display replicator 100 may beconfigured to power on in response to the display screen 102 beingtouched in a particular location or sense when an operator is in closeproximity to the screen and then power the electronic display replicatoron in reaction to the sensed presence. For example, the electronicdisplay replicator 100 may be configured to automatically power itselfon and/or otherwise activate itself when it senses the user is within acertain proximity (e.g., within 2 mm of the electronic display screen102). Alternatively, the electronic display replicator 100 may besuitably configured to sense a touched pattern on the display screen 102and power on in response.

Optionally, the plurality of control options 103, 104, 105 may bepositioned at any suitable location of the casing 101. For example, thecontrol option for the power switch 103 may be disposed anywhere alongthe perimeter of the casing 101. The control options may be disposed onthe surface of the casing 101.

Now referring to FIG. 2, optionally, the casing 101 may be configured toreceive the electronic display screen 102. The casing 101 may bedisposed around a perimeter of the electronic display screen 102 andwhile also enclosing the plurality of electronic components within thecasing 101.

The electronic display screen 102 may comprise any suitable system ordevice configured to display images, information, and/or data. Forexample, the electronic display screen 102 may comprise a thin-filmdisplay device such as an LCD (e.g., TFT LCD) or AMOLED screen. Theelectronic display screen 102 may comprise a rigid and/or flexibledisplay. The electronic display screen 102 may be configured to displayinformation and/or data in a variety of configurations. For example, theelectronic display screen 102 may be configured to display informationand/or data in standard definition, high definition, ultra-highdefinition, and/or any other suitable format. The electronic displayscreen 102 may be configured to display information in color, grayscale,black and white, and/or anything in between.

The electronic display screen 102 may comprise any suitable system ordevice configured to receive inputs. For example, the electronic displayscreen 102 may be configured with capacitive touchscreen capabilitiessuch that the electronic display screen 102 may receive inputs viaphysical manipulation of the electronic display screen 102. Theelectronic display screen 102 may also comprise other touchscreencapabilities such as resistive touch and the like.

The electronic display screen 102 may comprise any suitable size ordimension. The electronic display screen 102 may be sized to provide theuser with a larger display/viewing area than would otherwise be providedby the primary portable computing device's 201 native display screen. Itis understood by those in the art that screen sizes are typicallymeasured diagonally across the screen. For example, the electronicdisplay screen 102 may optionally comprise a 9.7-inch display and mayoptionally have an aspect ratio of 16 by 9. By way of further example,the electronic display screen 102 may optionally comprise a 10.07-inchdisplay and may optionally have an aspect ratio of 16 by 10, with anactive area of 216.96 mm by 135.6 mm. The electronic display screen 102may be configured to display data and/or information in any suitableresolution and/or aspect ratio. For example, aspect ratios of theelectronic display screen 102 and/or primary portable computing device301 display may further comprise 1:1, 5:4, 4:3, 3:2, 8:5, 5:3, 16:9,16:10, 17:10. 1.48:1, 1.85:1 and/or other aspect ratios (and theirinverse). The resolution may comprise standard definition, highdefinition, ultra-high definition 4K, and/or the like.

Additionally, the form factor of the electronic display screen 102and/or the electronic display replicator 100 may provide a user withenhanced capabilities and features not otherwise provided by the primaryportable computing device alone 201. For example, the larger form factorof the electronic display screen 102 may be easier to use as it providesthe user with an increased surface area to perform tasks, provide input,view data, and/or the like. Disposing the various electronic componentswithin a larger form factor case may provide the electronic displayreplicator 100 with enhanced cooling/heat reduction capabilities becausethe various electronic components do not have to be configured to be asclose in proximity as would be required in a smaller form factor such asthose found in the primary portable computing device 301. Furthermore,in certain optional configurations, the electronic display replicator100 does not require all of the same components found within the primaryportable computing device 301 because in certain optional configurationsthe electronic display replicator 100 is not used to perform as complexprocessing or computing functions as the primary portable computingdevice 301. For example, in certain optional configurations, the exampleprimary portable computing device 301 is used to perform all nativefunctioning allowing the electronic display replicator 100 to act solelyas a larger display system for the primary portable computing device301. By having fewer components, such configurations the electronicdisplay replicator 100 may be more lightweight, provide a longer lastingbattery life, thinner/less bulky, more reliable, and easier to operatecompared to a normal tablet computing device and/or smartphone. However,other configurations of the electronic display replicator 100 mayinclude robust processing power and specialized graphic processingdevices, may have greater processing capabilities than a conventionalcell phone, and may be configured to perform complex computing andprocessing functions. For example, the electronic display replicator 100may optionally include a multicore graphics processing unit configuredto decode and render video content, and configured to performcomputationally intensive transform, clipping, and lightingcalculations. The electronic display replicator 100 may be heavier orlighter, provide a longer or shorter battery life, be thicker orthinner/less bulky, more or less reliable compared to a conventionaltablet computing device and/or smartphone. The electronic displayreplicator 100 may be configured to charge the portable computing device201 via a wired connection (e.g., USB-C, microUSB, Lightning, etc.)and/or a wireless connection (e.g., via induction charging).

Optionally, the electronic display replicator 100 may be configured witha processor 108, a graphics card or circuit, and a memory 109. Theprocessor 108 may comprise any suitable device configure to provideprocessing capabilities. The processor 108 of the electronic displayreplicator 100 may optionally be configured to operate only the primaryportable computing device 301 and its various capabilities/features. Forexample, the processor 108 of the electronic display replicator 100 maybe configured to analyze, determine, and/or otherwise parse thedata/information related to the primary portable computing device 301 todetermine a graphical user interface to display to a user operating theelectronic display replicator 100 and/or the electronic display screen102.

The processor 108 of the electronic display replicator 100 may beconfigured to receive user inputs (e.g., touch inputs via a finger orstylus) via the electronic display screen 102 and operate the primaryportable computing device 301 using the reproduced graphical userinterface of the electronic display replicator 100. In such aconfiguration, the electronic display screen 102 may optionally actsimply an extension of the primary portable computing device 301providing a larger viewable area than could otherwise be displayed onthe primary portable computing device 301 itself. Optionally, theelectronic display replicator 100 may execute applications that areinstalled on the primary portable computing device 301 or theapplications may be executed by a remote server, or the applications maybe executed using a combination of the primary portable computing device301, the electronic display replicator 100, and a remote server, asdescribed in greater detail elsewhere herein.

The processor 108 may be configured to process data received via thewireless communication channel 302. For example, when the wirelesscommunication channel 302 is established, data from the primary portablecomputing device 301 may be transmitted to the electronic displayreplicator 100 for display. The processor may be configured to processthe data to determine whether an action needs to be taken on theelectronic display replicator 100 and/or the primary portable computingdevice. The memory 109 may comprise any suitable device configured toprovide memory capabilities and/or operate in conjunction with theprocessor 108 to provide processing/memory capabilities to theelectronic display replicator 100.

Referring now to FIGS. 2 and 3, optionally, the electronic displayreplicator 100 may be configured with a near-field communication (NFC)module 110. The NFC module 110 may comprise any suitable system ordevice configured to create, facilitate, operate, and/or manage awireless communication channel 302 between the electronic displayreplicator 100 and a primary portable computing device 301. The wirelesscommunication channel 302 may comprise any suitable connectionconfigured to relay operational commands and/or data between theelectronic display replicator 100 and the primary portable computingdevice 301. Optionally, the NFC module is configured to operate over avery short range of the primary portable computing device 301 (e.g., 4cm or less, 6 cm or less, 10 cm or less).

The electronic display replicator 100 may optionally comprise a controlsystem 113 configured to operate in conjunction with a NFC module 110, asuite of software 111, and various other components in creating,facilitating, operating, and/or managing the wireless communicationchannel 302. The control system 113 and/or the suite of software 111 maybe configured to establish a wireless communication channel between theelectronic display replicator 100 and the primary portable computingdevice 301 to facilitate the transposition and/or reproduction of datafrom a native display screen of the primary portable computing device301 onto the electronic display screen 102. For example, the electronicdisplay replicator 100 may be configured to receive an input via theelectronic display screen 102 to create a wireless communication channel302 with the primary portable computing device 301. The NFC module 110may communicate this request to the suite of software 111. The suite ofsoftware 111 may then be configured to create the wireless communicationchannel 302 between the electronic display replicator 100 and theprimary portable computing device 301.

Optionally, the suite of software 111 may comprise any softwareconfigured to operate on, or in conjunction with, an electronic device,network, and related components. For example, the suite of software 111may be configured to establish the wireless communication channel 302.In another example, the suite of software 111 may be further configuredto sense user inputs, transmit the sensed inputs to the primary portablecomputing device 301, transmit/receive information/data, processinformation/data, and/or perform other tasks related to the operation ofthe electronic display replicator 100.

Optionally, the wireless communication channel 302 may comprise anysuitable communication channel between the primary portable computingdevice 301 and the electronic display replicator 100. For example, thewireless communication channel 302 may comprise any suitable radiofrequency such as those used for Bluetooth® (e.g., Bluetooth® class 1,2, Bluetooth 4.1, Bluetooth LE, or other variant), WIFI, 802.11 and/orthe like. The wireless communication channel 302 may be configured to bebidirectional such that information may travel back and forth from theprimary portable computing device 301 and the electronic displayreplicator 100. The wireless communication channel 302 may be configuredto facilitate the transmission of various types of data including, butnot limited to, video, audio, configuration settings, graphical userinterface data, security data (e.g., passwords, cryptography keys,biometric data, etc.), environmental data, sensor data, user inputs,and/or the like. The wireless communication channel 302 may beconfigured with a bandwidth sufficient to accommodate the transmissionof the various data types. For example, larger bandwidth may be requiredto transmit video data compared to text data.

Optionally, the wireless communication channel 302 between theelectronic display replicator 100 and the primary portable computingdevice 301 may be established automatically, semi-automatically, and/ormanually. Optionally, the wireless communication channel 302 may beconfigured to exist only between the electronic display replicator 100and the primary portable computing device 301. In other words, thewireless communication channel 302 is configured for 1-to-1communication. For example, the wireless communication channel 302 maybe automatically created when the electronic display replicator 100 ispowered on and positioned within a certain geographical proximity to theprimary portable computing device 301.

Optionally, the wireless communication channel 302 may besemiautomatically created by the user using the electronic displayreplicator 100. For example, the electronic display replicator 100 maybe configured with a “create communication channel” option such as abutton/toggle/switch. When the “create communication channel”button/toggle/switch is activated, the electronic display replicator 100may attempt to automatically create a wireless communication channel 302with the primary portable computing device 301. For example, if theprimary portable computing device 301 has previously been used inconjunction with the electronic display replicator 100, either theprimary portable computing device 301 and/or the electronic displayreplicator 100 may be configured to store information/data regardingthat previous connection such that future wireless communicationchannels 202 between the two device may be created using the storedinformation/data.

The wireless communication channel 302 may also be created manually. Theelectronic display replicator 100 and/or the primary portable computingdevice 301 may be configured to receive information/data regarding eachdevice. For example, the electronic display replicator 100 may beconfigured to receive information/data regarding the primary portablecomputing device 301 such as the device identification data,login/password information, network identification data, the device WIFIaddress, the device Bluetooth address, the device IMEI, the deviceICCID, the device MEID, and/or the like. Similarly, the primary portablecomputing device 301 may be configured to receive similarinformation/data regarding the electronic display replicator 100 suchthat a wireless communication channel 302 may be created between the twodevices.

Now referring to FIG. 3, optionally, the primary portable computingdevice 301 may comprise any suitable electronic device such as a mobilephone, tablet, smart watch, vehicle display, and/or the like. Theprimary portable computing device 301 may be configured with anysuitable system or device configured to create a wireless communicationmedium with another electronic device (e.g., the electronic displayreplicator 100). For example, the primary portable computing device 301may be configured with its own NFC, Bluetooth, and/or WIFI modulesconfigured to create the wireless communication channel 302 inconjunction with another device (e.g., the electronic display replicator100). The primary portable computing device 301 may be disposed at aremote location relative to the electronic display replicator 100. Forexample, the primary portable computing device 301 may be disposedand/or otherwise be placed at a different geographical location relativeto the electronic display replicator 100 but still within a certaindistance proximity range to facilitate the creation of the wirelesscommunication channel 302 between the electronic display replicator 100and the primary portable computing device 301.

The wireless communication channel 302 may be configured to facilitatethe bidirectional flow of information and/or data back and forth betweenthe electronic display replicator 100 and the primary portable computingdevice 301. The wireless communication channel 302 may be utilized bythe electronic display replicator 100 and the primary portable computingdevice 301 to transmit operational commands between the two devices. Theoperational commands may comprise any type of command, instruction,operation, and/or the like configured to be executed on an electronicdevice. For example, an operational command may comprise operating afeature of one of the devices (i.e., increase/decrease volume, operatingthe capacitive touchscreen of either device, or the like).

Optionally, the wireless communication channel 302 may be configured toallow the primary portable computing device 301 to maintain a separatewireless network connection to a local area network that is distinct andindependent from the wireless communication channel 302 created betweenthe two devices. For example, the portable computing device 301 may becommunicatively linked to a local area network (such as WIFI) prior toestablishing the wireless communication channel 302 with the electronicdisplay replicator 100. After the wireless communication channel 302 hasbeen established, the primary portable computing device 301 may still becommunicatively connected to the local area network and transmit/receivedata/information via the local area network.

Optionally, when the wireless communication channel 302 has beenestablished between the electronic display replicator 100 and theprimary portable computing device 301, the primary portable computingdevice 301 may be configured to be operated via the electronic displayreplicator 100. In other words, the electronic display replicator 100may act as the controller for the primary portable computing device 301.The native display screen of the primary portable computing device 301may be turned off, put into a sleep/hibernation mode, and/or otherwisedisabled such that what was originally configured to be displayed on thenative display screen of the primary portable computing display device201 is now displayed in a larger format on the electronic display screen102 of the electronic display replicator 100. Turning off, disabling,hibernating, and/or otherwise disabling the native display screen of theprimary portable computing device 301 does not hinder the performanceand/or capabilities of the primary portable computing device 301.Rather, instead of displaying the information on the native displayscreen of the primary computing device 301, the information is insteadtransposed and/or reproduced on the electronic display replicator 100.

Optionally, the electronic display replicator 100 may be configured tooperate the primary portable computing device 301 according to inputsreceived via the electronic display replicator 100. The received inputsreceived by the electronic display replicator 100 may be transmittedand/or otherwise communicated to the primary portable computing device301 via the wireless communication channel 302 as discussed above. Thereceived inputs by the electronic display replicator 100 may beperformed by the primary portable computing device 301. For example,after the wireless communication channel 302 has been established, aninstruction received by the electronic display replicator 100 toincrease the volume setting may be communicated to the primary portablecomputing device 301 such that the volume produced by the primaryportable computing device 301 is increased.

Similarly, the user may select to open an app displayed on the displayscreen 102 and the control system 113 may transmit the instruction tothe primary portable computing device 301. The primary portablecomputing device 301 may then open the app which is then displayed onthe display screen 102 of the electronic display replicator 100. Theuser may then interact with the app via the display screen 102 and inplace of the screen of the primary portable computing device 301.

Optionally, the control system 113 may be configured to return thenative display device of the portable computing device 301 to an activestate in response to a user interaction with the primary portablecomputing device 301. The control system 113 may further cause theelectronic display replicator 100 to enter into a hibernation state whenthe primary portable computing device 301 is returned to an activestate. For example, the portable computing device 301 may comprise acell phone that is communicatively linked to the electronic displayreplicator 100 via the wireless communication channel 302. The primaryportable computing device 301 may receive a call which may require theuser's attention. The incoming call may be configured to be displayed onthe electronic display screen 102 of the electronic display replicator100. Optionally, the electronic display replicator 100 may temporarilydisable the wireless communication channel and/or disable the electronicdisplay replicator 100 from controlling the primary portable computingdevice 301 so that the user may operate the primary portable computingdevice 301 as it is normally configured to be operated (i.e., withoutthe electronic display replicator linked to it). Optionally instead, theelectronic display replicator 100 does not disable the wirelesscommunication channel and/or the electronic display replicator 100 fromcontrolling the primary portable computing device 301 so that the usermay operate the primary portable computing device 301 as it is normallyconfigured to be operated (e.g., to take or initiate a phone call on thedevice 301, to access an app directly on the device 301, etc.).

Referring back to FIGS. 1-3, optionally, the plurality of controloptions 103, 104, 105 of the electronic display replicator 100 may beutilized to control similar functions/devices of the primary portablecomputing device 301. The plurality of control options of the electronicdisplay replicator 100 may be mapped, linked to, associated with, and/orotherwise generally related to a similar function of the primaryportable computing device 301. For example, the electronic displayreplicator 100 may comprise a control option 104 for volume control.Adjusting the volume control on the electronic display replicator 100may cause the second device 201 to increase its volume.

Now referring to FIG. 4, the electronic display replicator 100 mayoptionally be configured to be paired and/or otherwise associated with aprimary portable computing device 301 over a dedicated wirelessconnection. The pairing and/or association process may comprise anysuitable system configured to pair and/or associate two devicestogether. For example, the electronic display replicator 100 and theprimary portable computing device may be configured to share betweeneach other any settings, configurations, access controls, network IDs,encryption, keys, and/or the like between each other to establish apairing.

The control system 103 may be configured to transmit a signal to theprimary portable computing device 301. The signal may be configured toactivate a pairing protocol between the electronic display replicator100 and the primary portable computing device 301. Once paired and/orassociated together, the electronic display replicator 100 and theprimary portable computing device 301 may store the pairing informationfor later use. For example, once the electronic display replicator 100and the primary portable computing device 301 have been paired togetherfor the first time, the two devices retain information regarding thatpairing/association such that the two devices do not have to undergo theprocess again in the future.

When the two devices are paired and/or otherwise associated with eachother, a wireless communication channel 302 may be established betweenthe two devices (302). The wireless communication channel 302 maycomprise any suitable system configured to establish a communicationchannel between two devices as discussed above. For example, thewireless communication channel 302 may comprise various wirelesscommunication protocols such as Bluetooth®, WIFI, 802.11, radiofrequency, and/or the like.

Optionally, after the wireless communication channel 302 has beenestablished, data and/or information may be transposed and/or reproducedfrom the primary portable computing device 301 to the electronic displayreplicator 100 over the wireless communication channel 302 (303). Thedata and/or information may comprise any data and/or informationregarding pictures, video, audio, and/or the like.

The data and/or information may also comprise data/information relatedto a data set comprising graphical user interface data. It is understoodto those skilled in the art that a graphical user interface may compriseany suitable system configured to display information to a user. Thegraphical user interface may comprise text, icons/graphics, formattinglayout, and the like. The graphical user interface data may beconfigured to allow the electronic display screen 102 to reproduce agraphical user interface meant to be displayed on the native displayscreen of the portable computing device 301. For example, the primaryportable computing device 301 may provide to the electronic displayreplicator 100 via the wireless communication network 202 a graphicaluser interface data set such that the processor 108 of the electronicdisplay replicator 100 may extract and/or otherwise retrieve from thegraphical user interface dataset data/information to replicate,transpose, and/or otherwise reproduce the same graphical interface onthe electronic display replicator 100 that would otherwise be configuredto be displayed on the primary portable computing device 301.

The electronic display replicator 100 may receive a plurality of userinputs (304). The received inputs may comprise user created inputs(e.g., a user using the capacitive touchscreen of the electronic displayreplicator 100) and/or the received inputs may comprise inputs receivedfrom another electronic device. The received inputs may be provided tothe processor 108 for processing to determine whether an action needs tobe taken on either the electronic display replicator 100 and/or theprimary portable computing device 301. The suite of software 111 may beconfigured to operate in conjunction with the processor 108. In oneconfiguration, the bulk of the processing may performed by the primaryportable computing device 301. The primary portable computing device 301may be configured to provide the processed data to the electronicdisplay replicator to transpose and/or otherwise reproduce.

Optionally, the plurality of user inputs received by the electronicdisplay replicator 100 may be configured to operate the primary portablecomputing device 301 (305). For example, if the supplemental displayscreen 100 received inputs to open an application, the input may betransmitted to the primary portable computing device 301 so that theprimary portable computing device 301 may open the application.

Further example architectures, processes, and features will now bedescribed.

FIG. 5 illustrates an example operational architecture, including asupplemental display device 502, a first computing device 504 (e.g.,which optionally may be the primary portable computing device 301), anda remote system 506, which may be a cloud-based server system. The firstcomputing device 504 may have a smaller display than the supplementaldisplay device 502 (although optionally, the supplemental display device502 may have the same size display or a smaller display than that of thefirst computing device 504). For example, the supplemental displaydevice 502 may have a display size of between 7.0-12.3 inches (orgreater) in the diagonal, and the first computing device 504 may have adisplay size of between 1.0-6.9 inches in the diagonal. The supplementaldisplay device 502, the first computing device 504, and the remotesystem 506 may communicate via one or more wireless and/or wiredinterfaces. The supplemental display device 502, the first computingdevice 504, and the remote system 506 may communicate over a wide areanetwork, such as the Internet. Optionally, the supplemental displaydevice 502 and/or the first computing device 504 may access the widearea network via an access point 510 if available. The supplementaldisplay device 502 and the first computing device 504 may alsocommunicate with each other via a local network, such as a WIFI Directnetwork and/or via Bluetooth®.

Optionally, different wireless interfaces may be used to communicatedifferent types of instructions or data. For example and withoutlimitation, the supplemental display device 502 and the first computingdevice 504 may include Bluetooth® (e.g., Bluetooth® class 1, 2,Bluetooth 4.1, Bluetooth LE (supporting Bluetooth® class 1, 2, and/orBluetooth 4.1), or other variant) and WIFI 802.11 radios/interfaces. Thesupplemental display device 502 and/or the first computing device 504may also include mobile communication radios/interfaces, such ascellular network radios/interfaces (e.g., 3G, 4G (e.g., LTE, HSPA,etc.), 5G or other variants of cellular technology).

Optionally, Bluetooth LE supporting Bluetooth 4.1 may be utilized toreduce or eliminate conflicts with the cellular radio (e.g., LTE radio)by coordinating the Bluetooth radio with the cellular radio so there isreduced or no overlap. Further, rather than utilizing a fixed timeoutperiod, Bluetooth 4.1 enables the supplemental display device 502 toutilize custom-specified reconnection timeout intervals to better managepower utilization and to enable automatic powering up and down based ona custom power plan.

The supplemental display device 502 may optionally include some or allof the following: one or more touch displays (e.g., a 2K, 4K, 8K, orother resolution display, where the display may be a resistive orcapacitive touchscreen, and may utilize LCD, OLED, AMOLED, or otherdisplay technology), a non-touch display, LED indicators, a CPU (e.g., a64 bit multicore processor), a motion coprocessor, a graphics processor,volatile memory, non-volatile memory, a Bluetooth modem, one or moreBluetooth antennas, a WIFI modem, one or more WIFI antennas, a cellularmodem, one or more cellular antennas, one or more front facing cameras,one or more rear facing cameras, an accelerometer, a tilt sensor, anorientation sensor, one or more speakers, one or more microphones, oneor more ambient light sensors, one or more biometric sensors and systems(e.g., fingerprint reader, iris reader, camera, face recognition system,voice recognition system, and/or the like) to unlock the supplementaldisplay device 502, make payments, and/or to access sensitive data(e.g., credit card information, account information, or the like), arechargeable battery, and/or a wireless recharging circuit (e.g., aninduction charging circuit). Optionally, the supplemental display device502 may provide virtual simultaneous dual band operation (e.g., 2.4 GHzISM Band (2.400-2.497 GHz)). The supplemental display device 502 may beconfigured to charge the first computing device 504 via a wiredconnection (e.g., USB-C, microUSB, Lightning, etc.) and/or a wirelessconnection (e.g., via induction charging).

The supplemental display device 502 may include an operating system(e.g., an Android-based operating system, and iOS-based operatingsystem, a Windows-based operating system, a macOS operating system, aLinux-based operating system, or the like). The supplemental displaydevice 502 may provide user access and control of the first device'soperating system and data connections. The supplemental display device502 may optionally include one or more physical ports. For example,supplemental display device 502 may include a physical port for chargingand/or for digital communication, such as a USB port (e.g., USB-C,microUSB, etc.), a Lightning port, etc. The supplemental display device502 may optionally include a memory port (e.g., an SD or microSD port)configured to removably receive a memory module (e.g., a non-volatile SDor microSD card).

The supplemental display device 502 may optionally include a physicalvolume control, a physical mute control, a physical screen rotationcontrol (e.g., to prevent or permit rotation of the content displayed onthe screen in response to the user rotating the supplemental displaydevice 502), and/or a physical power on/off control. In addition orinstead, some or all of the foregoing controls may be provided as “soft”controls accessed via a touch input device (e.g., a touchscreen).

As discussed in greater detail elsewhere herein, the supplementaldisplay device 502 may scale content from the first computing device 504so that the content is not blurred or chopped when rendered on thesupplemental display device 502 display. For example, optionally thesupplemental display device 502 may render raw content data from thefirst computing device 504 using the lesser of the supplemental displaydevice display resolution and the resolution of the raw content data, sothat the resolution of the first computing device 504 does not controlthe final viewed resolution on the supplemental display device 502.Thus, unlike conventional mirroring techniques, raw data may betransmitted bi-directionally between the supplemental display device 502and the first computing device 504.

The supplemental display device 502 may be configured to automaticallymute its speakers in response to detecting (via a message transmittedfrom the first computing device 504 to the supplemental display device502) that the user is on a call on the first computing device 504 and/orthat the user is playing audio content on the first computing device504.

The supplemental display device 502 may optionally be configured toprovide its services to a large variety of other computing device types,including without limitation tablets and phones using the Androidoperating system, the iOS operating system, the Windows operatingsystem, and the like. By way of further example, the supplementaldisplay device 502 may optionally be configured to provide its servicesto wearables (e.g., smart watches, medical devices, and the like), otherInternet of Things (IoT) devices, car displays, appliances, and thelike.

As described in greater detail elsewhere herein, optionally thesupplemental display device 502 may utilize absolute positioning withrespect to user touch inputs to avoid the technical problems of relativepositioning (where the first computing device's standard mouse driver isused to read the supplemental display device 502 as a mouse HID), wherethe user would have to indicate a 0,0 position at the beginning of eachsession so that the supplemental display device 502 can calibrate userinputs.

Communication and synchronization between the supplemental displaydevice 502 and the first computing device 504 may be managed usingcommunication and synchronization applications respectively installed onthe supplemental display device 502 and the first computing device 504.The communication and synchronization application installed on thesupplemental display device 502 may provide different functionality thanthat of the communication and synchronization application installed onthe first computing device 504 and may include different code. Forexample, as described in greater detail elsewhere herein, acommunication and synchronization application may enable thesupplemental display device 502 to unlock the first computing device504, set up communication channels with the first computing device 504,control the first computing device display, launch applications on thefirst device 502, access the user interfaces of launched applicationsfrom the first user device 502, access data being received by suchapplications, display and/or play content being received by suchapplications, display app icons from the first computing device 504,display interfaces of the first computing device 504, and/or the like.

The communication and synchronization applications optionally may bedownloaded from an application store or the applications may bepreinstalled on either or both the supplemental display device 502 andthe first computing device 504. For example, the communication andsynchronization application or aspects thereof may be part of theoperating system of the supplemental display device 502. Thecommunication and synchronization application or aspects thereof mayoptionally be in the form of firmware. Optionally, certain services maybe implemented in firmware (e.g., a haptic service, a sound service,and/or the like) and other services may be implemented in one or moreinstalled software applications. The functionality of the communicationand synchronization application may be divided among multipleapplications.

Where the supplemental display device 502 uses a different operatingsystem than the first computing device 504, and the communication andsynchronization applications (or updates thereof) are to be downloaded,the supplemental display device 502 may download the application (orupdates thereof) from a different application source (e.g., applicationstore) than the first computing device 504. For example, if thesupplemental display device 502 utilizes the Android operating system(which may be a customized/forked version of the Android operatingsystem), the communication and synchronization application may downloadthe application and/or updates to the application from an Androidapplication store, while if the first computing device 504 is utilizesan iOS operating system, the communication and synchronizationapplication may download the application and/or updates to theapplication from an Apple application store. Optionally, thecommunication and synchronization application for the first computingdevice 504 may first be downloaded or otherwise provided to or installedon the supplemental display device 502, which may then transmit thecommunication and synchronization application to the first computingdevice 504 for installation thereon (e.g., via Wi-Fi Direct, Bluetooth,or otherwise).

The communication and synchronization application on the first computingdevice 504 may monitor one or more communication channels of the firstcomputing device 504 (e.g., Bluetooth, Wi-Fi Direct, cellular, and/orthe like), and detect if an instruction (which may correspond to usertouch inputs received via the supplemental display device 502) has beenreceived from the supplemental display device 502. The communication andsynchronization application on the first computing device 504 may thencause such instruction to be executed by the first computing device 504.

Optionally, in order to provide secure communication, an asymmetriccryptographic algorithm may be utilized to ensure that information isnot improperly accessed as it is being transmitted between the firstcomputing device 504 and the supplemental display device 502. Forexample, the communication and synchronization application on the firstcomputing device 504 may generate two keys. One of the keys may besecurely stored on the first computing device 504 and the second key maybe wirelessly transmitted to the supplemental display device 502.

The first key may be composed of a modulus and an encryption exponent,and the second key may contain the same modulus as the first key with adecryption exponent which may be securely stored on the supplementaldisplay device 502. Thus, the two keys may have a common element in themodulus, but may have different elements in their exponents.

After the key exchange has been performed, the supplemental displaydevice 502 may utilize the keys to later identify and verify theidentity of the first computing device 504 without having new keysgenerated.

To further enhance security, convenience, and flexibility, thesupplemental display device 502 may optionally establish and utilize asingle radio hop communication connection (e.g., Wi-Fi Direct) with thefirst computing device 504 such that the supplemental display device 502and the first computing device 504 operate in a security domain withoutuse of a separate wireless access point. Optionally, a Wi-Fi Directconnection/pairing may be established using a Bluetooth or NFCcommunication. Optionally, the Wi-Fi Protected Access II (WPA2) protocol(or other security protocol) may be utilized but may be managedseparately from the security system in an access point-based network(e.g., a home, enterprise, or hotspot access point).

For example, the supplemental display device 502 may execute a service,such as mDNS (multicast Domain Name Service) or other resolving service,which may be a zero-configuration service, that resolves host names toIP addresses (e.g., within networks that do not include a local nameserver), to perform a search to locate the first computing device 504.The supplemental display device 502 may then access and verify theencrypted keys to ensure that the first computing device 504 has beenpreviously registered by the supplemental display device 502.

Optionally, certain data, such as publically available data not specificto the user and not confidential in nature, streamed from the Internetto the first computing device 504 and from the first computing device504 to the supplemental display device 502 will not be so encrypted.Optionally, such data may be encrypted using simpler cryptography, suchas salt cryptography where random data may be used as an additionalinput to a one-way function that hashes data.

Optionally, access to the supplemental display device 502 may be madefurther secure via password (e.g., an alphanumeric password) and/orbiometric protection. For example, the supplemental display device 502may enable a user to setup a password and/or biometric input which arerequired to unlock the supplemental display device 502. Optionally,two-factor authentication may be provided, wherein two authenticationfactors are used to authenticate a user (e.g., at least two of thefollowing: a password, a PIN, a shared secret, a security token, anotherdevice (e.g., an ID card or laptop), or biometrics) in order to permitaccess the supplemental display device 502.

To further enhance security, the communication and synchronizationapplication on the first computing device 504 may include a disablecontrol that when activated causes the communication and synchronizationapplication to prevent the supplemental display device 502 fromcommunicating with or accessing the first computing device 504.Activation of the disable control may be stored in memory. This ensuresthat even if the supplemental display device 502 is lost or stolen, andeven if the person who found or stole the supplemental display device502 is able to penetrate the password and/or biometric protectionfeatures of the supplemental display device 502, and is within wirelessrange of the first computing device 504, the supplemental display device502 would still not be able to control or access data from the firstcomputing device 504.

The communication and synchronization application on the first computingdevice 504 may determine which applications are installed on the firstcomputing device 504, and may optionally generate a custom userinterface (which may include one or more screens) that displays theapplication icons in a grid of columns and rows. Optionally, the usermay modify the placement of icons on the custom user interface generatedby the communication and synchronization application (e.g., by draggingand dropping icons to new locations).

The layout of the application icons presented by the custom userinterface may be different than, or the same as, the native layout ofapplication icons. Thus, optionally, rather than viewing the native userinterface of rows and columns of application icons, the user is viewinga user interface generated by the communication and synchronizationapplication on the first computing device 504. This enables thesupplemental display device 502 to have full access to the firstcomputing device 504 display (e.g., for transferring raw data URLs andinternal commands). For example, during initial setup, when the firstcomputing device 504 and the supplemental display device 502 are paired,the communication and synchronization application on the first computingdevice 504 may access such information and pass it back to thesupplemental display device 502 (e.g., via Bluetooth).

Optionally, the layout rendered on the supplemental display device 502may be the same layout displayed on the user interface generated by thecommunication and synchronization application on the first computingdevice 504 (e.g., as a default). If the icons rendered on the largerdisplay of the supplemental display device 502 are of the same size asthe icons rendered on the smaller display of the first computing device504 and if both devices have the same icon layout, there will beadditional space available on the larger supplemental display devicedisplay that is not used to display applications icons. Optionally, theadditional display space on the supplemental display device 502 may beused to display notifications, video playback, or other entertainment orwork related data.

Optionally, a user interface may be provided via the supplementaldisplay device 502 that enables a user to specify the grid size (e.g.,in terms of the number of columns and row) that is to be used to displayapplication icons on the supplemental display device 502. For example,the grid on the first computing device 504 may 4 columns and 6 rows (andso capable of displaying 24 application icons). However, the userinterface may enable the user to specify (via respective fields or via agrid drawing tool) that a grid of 8 columns and 10 rows is to be used todisplay application icons on the supplemental display device 502 (and socapable of displaying 80 application icons). The supplemental displaydevice 502 may then display the icons in a grid according to the userspecification. Optionally, the user interface may enable the user tospecify which application icons are to be displayed in a dock area,where the applications in the dock area is continuously displayed evenwhile the user is swiping through different screens of applicationicons.

Optionally, a settings user interface is provided where the user canspecify if character fonts and/or icons from the first computing device504 are to be resized (e.g., to larger size or smaller size) on thesupplemental display device 502. Optionally, the settings user interfacemay enable the user to specify that the character fonts and/or iconsheights/widths are to be sized proportional to the ratio of the heightof the display of the supplemental display device 502 to the height ofthe display of the first computing device 504 and the width of thedisplay of the supplemental display device 502 to the width of thedisplay of the first computing device 504. Optionally, the settings userinterface may enable the user to specify a scaling percentage of thecharacter fonts and/or icons. For example, the settings user interfacemay enable the user to specify a scaling percentage of 25%, 50%, 100%,125%, 150%, 200%, 400% and/or other percentage via a menu of scalingpercentages. The settings user interface may optionally enable the userto specify a scaling percentage by manually entering a percentage numberinto a scaling field. The user scaling inputs may be stored in memoryand utilized for scaling icons and fonts displayed on the supplementaldisplay device 502.

The supplemental display device 502 may optionally use the Bluetoothinterface to: communicate an unlock instruction to the first computingdevice 504; for pairing with the first computing device 504; fortransmitting streaming commands (e.g., start, stop, pause, rewind, fastforward) to the first computing device 504; for receiving informationregarding which applications are installed on the first computing device504; for communicating a communication environment of the firstcomputing device 504 (e.g., an indication as to whether the firstcomputing device is connected to a cellular network, a WIFI accesspoint, or the like); for enabling a hotspot function on the firstcomputing device 504 (so that the supplemental display device can accessthe Internet via the first computing device 504 cellular connection);and/or to communicate audio-only content.

The supplemental display device 502 may use optionally the WIFIinterface to: communicate video data with the first computing device 504(e.g., stream video data or transmit video files from the firstcomputing device 504 to the supplemental display device 502);communicate photographs with the first computing device 504; transmitraw data URLs (the portion of the URL following the domain information)for a video player; and/or transmit user touch inputs received at thesupplemental display device 502 to the first computing device 504. Thesupplemental display device 502 may also use the WIFI interface tocommunicate with (e.g., via an access point) the remote system 506 overone more networks 506.

Communications from the supplemental display device 502 may be receivedand processed by the communication and synchronization application onthe first computing device 504. It is understood that certain data orinstructions described above as being communicated via Bluetooth may becommunicated via WIFI and that certain data or instructions describedabove as being communicated via WIFI may be communicated via Bluetooth,although there may be adverse impacts.

When needed, data received over two or more communication channels(e.g., Bluetooth and WIFI) may be synchronized. For example, audio datareceived via the Bluetooth interface may be synchronized with a videostream received via the WIFI interface. When the supplemental displaydevice 502 and first computing device 504 are paired, the date/time ofthe supplemental display device 502 may be synchronized with that of thefirst computing device 504. The communication and synchronizationapplication on the first computing device 504 may associate a time stampwith a given video frame being transmitted from the first computingdevice 504 to the supplemental display device 502. Time stamps may alsobe associated with corresponding “frames” of the audio data. The audioand/or video data and associated data stamps may be received by thesupplemental display device 502. The audio data and/or the video datamay be buffered in memory on the supplemental display device 502. Thesupplemental display device 502 may use the timestamps to synchronizeplayback of the audio data and the video data. Optionally, the audiodata and video data are not merged into a single stream during playbackas such merging may unnecessarily utilize additional computing resourcesand may introduce lags in the video/data playback.

With respect to audio data, certain operating systems, such as theAndroid operating system, prevent third party application from grabbingaudio data on the device (e.g., an Android device). For example,although the Android operating system provides a CAPTURE_AUDIO_OUTPUTpermission, it may not be used by third-party applications. To overcomethis challenge, a Bluetooth speaker Human Input Device (HID) module isprovided which enables the communication and synchronization applicationon the first computing device 504 to obtain audio data on the firstcomputing device 504 and pass the audio data to the supplemental displaydevice 502 without requiring the use of the CAPTURE_AUDIO_OUTPUTpermission or other permissions.

Optionally, unlike conventional mirroring, the supplemental displaydevice 502 may be configured so that content displayed by thesupplemental display device 502 is orientated (e.g., in landscape orportrait format) based on the orientation of the supplemental displaydevice 502 (as detected using orientation sensors), and not theorientation of the first computing device 504. For example, content onthe supplemental display device 502 may orientate from portrait tolandscape (or vice versa) based on the detected user rotation of thesupplemental display device 502 being held by the user, entirelyindependent of the orientation of the user's first computing device 504.By way of illustration, if the supplemental display device 502 is beingheld in a landscape mode, content displayed on the supplemental displaydevice 502 will be formatted in landscape mode, even if the firstcomputing device 504 is oriented in portrait mode. Thus, the orientationof the displayed interfaces of the supplemental display device 502 andthe first computing device 504 may be decoupled from each other.

As noted elsewhere herein, optionally, the layout of the application(“app”) icons presented on the supplemental display device 502 mayarranged so as to match the layout of the application icons on the firstcomputing device 504. Optionally instead, in order to take advantage ofthe relatively larger display of the supplemental display device 502,the application icon layout rendered on the supplemental display device502 may be different than that of the custom user interface on the firstcomputing device 504. For example, a given row of application iconsdisplayed on the supplemental display device 502 may include moreapplication icons than a row of application icons displayed on the firstcomputing device 504. The supplemental display device 502 may optionallybe configured to display more rows and columns of application icons thanthe first computer device 504. Thus, if it takes several screens todisplay all the application icons on the first computing device 504, itmay take fewer screens to display those same application icons on thesupplemental display device 502, requiring less swiping of screens by auser to access a desired icon. Optionally, the size of the applicationicons displayed on the supplemental display device 502 may be the sameas the size of the application icons displayed on the first computingdevice 504 to avoid distortions or pixilation.

In general, the communication and synchronization application may obtainapp icons/text, cookies, and/or passwords from the first device and maythen store them in supplemental display device memory for utilization bythe supplemental display device 502. Optionally, the communication andsynchronization application installed on the first computing device 504detects any new apps installed on the first device, transmits the appicon, text, and password (as appropriate or available) to thesupplemental display device 502, and the supplemental display device 502may then automatically display the app icon and/or text at acorresponding location, and use the app password as needed (e.g., toaccess a user account or data associated with the app).

Optionally, the supplemental display device 502 is configured to preventan end user from downloading third party applications to thesupplemental display device 502, thereby further ensuring thereliability and security of the supplemental display device 502 andreducing the utilization of non-volatile memory on the supplementaldisplay device 502.

As described in greater detail elsewhere herein, the supplementaldisplay device 502 may instruct the first computing device 504 toexecute applications hosted on the first computing device 504, or mayexecute the application (e.g., a copy of the application) on thesupplemental display device 502, or may instruct the remote system 506to execute the application (e.g., a copy of the application or a versionthereof). The supplemental display device 502 may detect user inputs(e.g., user touches on the on the supplemental display device touchdisplay), determine what icon or icon control or menu item the usertouched and/or the location of the touch, transmit correspondinginformation to the first computing device, which may then execute thecorresponding action (e.g., launch a user select app or execute a userselection of a menu item or command).

Certain techniques may be utilized to reduce latency with respect toperforming certain functions. For example, the supplemental displaydevice 502 may include a data buffer which may be used to buffer datafrom the supplemental display device 502 to ensure that data isavailable as needed for the supplemental display device 502 to render,play, and/or process the data. Further, by receiving and processing rawdata from the first computing device 504 (where the raw data is receivedby the first computing device 504 from another source, such as theInternet), rather than relying on the first computing device 504 tofirst process the data (e.g., to scale, decode or decrypt the data), thesupplemental display device 502 has lower latency access to the data. Inaddition, if the supplemental display device 502 determines that thedata rate/available bandwidth of data communication from the firstcomputing device 504 is reduced (e.g., because of an increasing distancebetween the two devices or interference), the supplemental displaydevice 502 may instruct the first computing device 504 to reduce thevideo quality being transmitted to the supplemental display device 502to reduce the amount of data that needs to be transmitted to therebyreduce or entirely eliminate interruptions in video playback.

For example, the connection speed of the wireless network between thefirst computing device 504 and the supplemental display device 502 mayoptionally be tested as follows. The communication and synchronizationapplication on the first computing device 504 may transmit one or moretest bytes to the supplemental display device 502 in association with atime stamp. The supplemental display device 502 may determine at whattime it received the test byte (using its internal clock), compare thereceipt time with the timestamp, and determine the ping/delay time. Ifthe ping time is unacceptable (e.g., greater than a threshold), thequality of the video being transmitted may be reduced so as to satisfythe threshold.

Optionally, to further reduce latency, if the communication andsynchronization application on the first computing device 504 cannotdetermine or detect the exact locator (e.g., URL) of content (e.g.,video content) the user is requesting via a webpage being accessed bythe first computing device 504 (e.g., because the URL is malformed),where the content is to be rendered on the supplemental display device502, the URL of the webpage currently being accessed may be transmittedto the remote system 506. For example, the communication andsynchronization application on the first computing device 504 maytransmit the webpage URL directly to the remote system 506, or, if thesupplemental display device 502 is connected to an access point, thefirst computing device 504 may transmit the webpage URL to thesupplemental display device 502, which may in turn transmit the webpageURL to the remote system 506. The remote system 506 may parse thecorresponding webpage, determine the exact URL of the content, andtransmit the exact URL to the supplemental display device 502 (directlyor via the first computing device 504). The supplemental display device502 may utilize the URL from the remote system 506 to request the videocontent. For example, if an access point is available via which thesupplemental display device 502 can access the Internet, thesupplemental display device 502 may access the content at the exact URLvia the access point, without having to route the request to orotherwise utilize the first computing device 504. If an access point isnot available, but the first computing device 504 has Internet accessvia its cellular radio, then the first computing device 504 may accessthe video content at the exact URL, and as it receives the videocontent, stream the video content to the supplemental display device502, optionally without processing the video content via a video codec.

Where the first computing device 504 can determine the exact content URLand the supplemental display device 502 can access the Internet via anaccess point, the first computing device 504 may transmit the raw dataURL to the supplemental display device 502. The supplemental displaydevice 502 may utilize the raw data URL to access the video content atthe raw data URL directly, without having to route the request to orotherwise utilize the first computing device 504. The requested videocontent may be streamed to the supplemental display device 502 which mayrender and playback the video content (e.g., using the native aspectratio of the video content). This technique overcomes drawbacksassociated with mirroring the playback of video content from a sourcedevice to a sink device. Such conventional mirroring techniques oftenleave one or more blank boarders around the video when rendered on thesink device. By contrast, the described technique optionally allows thevideo to be played back in full screen mode, without such borders (e.g.,where the native aspect ratio of the video content matches the aspectratio of the display of the supplemental display device 502).Optionally, if the native aspect ratio of the video content does notmatch the aspect ratio of the display of the supplemental display device502, the video content will be scaled so as to be displayed in fullscreen mode on the supplemental display device 502. Optionally, acontrol may be provided via which the user can specify whether or notvideo content is to be scaled so as to be displayed in full screen modeon the supplemental display device 502, and the setting may be used forfuture playback of video content by the supplemental display device 502.

In order to conserve power and extend battery life, the supplementaldisplay device 502 may determine, based on determined activity on thefirst computing device 504 and/or the supplemental display device 502,as to whether the first computing device 504 may be placed in a reducedpower consumption mode. For example, if the supplemental display device502 determines that it is playing a video, and determines, frominformation provided by the communication and synchronizationapplication on the first computing device 504, that the user has notprovided any user input on the first computing device 504 (e.g., via thefirst computing device 504 touchscreen) for at least a threshold periodof time, the supplemental display device 502 may infer that the user itnot utilizing the display of the first computing device 504 and thesupplemental display device 502 may command the first computing device504 to hibernate or reduce the brightness of its display. Thehibernation command may also cause the first computing device 504 toreduce processing speed, turn off background applications, and/or takeother power reduction actions that the first computing device 504 hasbeen set up to take in response to a hibernation command.

The supplemental display device 502 may transmit a hibernate command ora reduce brightness command (e.g., if the first computing device 504does not permit a remote display hibernate command, such as is the casein iOS devices) to the first computing device 504 using Wi-Fi Direct orBluetooth (although other communication channels may be used). Thecommunication and synchronization application on the first computingdevice 504 may detect the command, and accordingly cause the display tohibernate or to reduce its brightness (e.g., to zero or other level)and/or take other hibernation actions to thereby conserve power.

Optionally, the hibernation command may be automatically issuedsubstantially immediately (e.g., within 1 second, within 30 seconds,within 60 seconds) after the supplemental display device 502 establishesa connection with the first computing device 504. Optionally, thesupplemental display device 502 may wake up the first computing device504 or change the brightness of the first computing device display viaan instruction transmitted using its Bluetooth radio (although othercommunication channels may be used).

Optionally, the user may manually wake up or increase the brightness ofthe first computing device display and otherwise stop the hibernationprocess (e.g., by touching the first computing device touchscreen, bypressing a home or power control, etc.). Optionally, when thesupplemental display device 502 changes the brightness of the firstcomputing device display (e.g., from off or lowest brightness tovisible), the supplemental display device 502 may utilize a globalbrightness value accessed from the first computing device 504 to set thefirst computing device display brightness. The communication andsynchronization application on the first computing device 504 may accessand store (e.g., in the User Settings class) the global brightness valueafter launching for later use in restoring display brightness whenending a hibernation state.

Optionally, the communication and synchronization application on thefirst computing device 504 may automatically mute the speakers of thefirst computing device 504 upon launch of the communication andsynchronization application on the first computing device 504 or inresponse to connecting to the supplemental display device 502.Optionally, a user-accessible setting control may be provided via whicha user can specify that the speakers of the first computing device 504are not to be so automatically muted. Optionally, the communication andsynchronization application on the first computing device 504 mayautomatically unmute the speakers in response to being disconnected fromthe supplemental display device 502. Optionally, the communication andsynchronization application on the first computing device 504 enablesthe user to manually unmute the speakers of the first computing device504 (e.g., by activating a volume control or unmute control on the firstcomputing device 504).

The supplemental display device 502 may optionally be configured to becompatible with a large variety of computing devices, such as a widevariety of iOS-based phone models and a wide variety of Android-basedphone models from different manufacturers, including those withdifferent versions of iOS and Android operating systems and differentversions of firmware. For example, the supplemental display device 502may include multiple streaming applications, including a streamingapplication compatible with Windows-based device, a streamingapplication compatible with Samsung devices, and a streaming applicationcompatible with Android-based devices. The supplemental display device502 may determine what operating system the first computing device 504is using and the device type, access a table that stores a matrixmapping operating systems and device types to applications, and utilizethe appropriate applications in communicating data and instructions withthe first computing device 504. By way of illustration, the supplementaldisplay device 502 may utilize Miracast to capture display content onthe first computing device 504, if available on the first computingdevice 504. If Miracast is not available and if the first computingdevice 504 is Android-based, the supplemental display device 502 mayutilize the Media Projection API to capture display content on the firstcomputing device 504. The UDP (User Datagram Protocol) protocol may beused to stream content from the first computing device 504 to thesupplemental display device 502.

By way of further example, the matrix may map various combinations ofmanufacturer, manufacturer model, manufacturer manufacturing date (wheretwo of the same model device from the same manufacturer, with the sameoperating system, may have different firmware), operating system, and/oroperating system version to corresponding applications and instructions.The foregoing information may be transmitted by the first computingdevice 504 to the supplemental display device. This techniqueadvantageously enables the supplemental display device 502 to becompatible with a wide variety of device models and versions of modelsfrom a given manufacturer and a wide variety of different devices fromdifferent manufacturers. For example, as described elsewhere herein, thesupplemental display device 502 may utilize different commands to causedifferent types of computing devices to enter into a reduced powerconsumption state.

Dual mode functionality may be provided enabling the user to use anapplication on the first computing device 504 (e.g., to place a phonecall) and simultaneously use an application (e.g., a social networkingapplication or book reader application) on the supplemental displaydevice 502. Optionally, the first computing device 504 may execute andperform the processing for the application being utilized on the firstcomputing device 504. Optionally, a combination of the first computingdevice 504, the supplemental display device 502, and the remote system506 may be utilized to execute and perform the processing for theapplication being utilized on the supplemental display device 502.

Optionally, the communication and synchronization application on thefirst computing device 504 enables a reverse operation, where userinputs on the first computing device 504 are used to determine what isdisplayed on one or on multiple supplemental display devices at the sametime. This function enables a presenter to manage, from a first device,what is displayed on multiple supplemental display devices provided toan audience. For example, the presenter may be a teacher and theaudience may comprise students. By way of further example, the foregoingfunction would enable a parent playing a movie on the parent's device(e.g., a phone) to cause the movie to be rendered on children'ssupplemental display devices. Optionally, a given supplemental displaydevice may similarly transmit content (e.g., received from the firstcomputing device 504) to multiple other user devices that havecommunication and synchronization application installed thereon so thatthe multiple other user devices display the content at the same time.

Optionally, the supplemental display device 502 may be used to relayvoice instructions to the first computing device 504. By way ofillustration, the supplemental display device 502 may determine, usingthe communication and synchronization application installed on the firstcomputing device 504, what type of natural language processing serviceor other voice responsive personal assistant service is utilized by thefirst computing device 504. For example, if the first computing device504 is determined to be an iOS-based device, the supplemental displaydevice 502 may determine that the first computing device 504 utilizesthe Ski® natural language processing service. If the first computingdevice 504 is determined to be an Android-based device, the supplementaldisplay device 502 may determine that the first computing device 504utilizes the Google Now natural language processing service. By way offurther example, if the first computing device 504 is determined to be aWindows-based device, the supplemental display device 502 may determinethat the first computing device 504 utilizes the Microsoft Cortana®natural language processing service. Based on the determined naturallanguage processing service, the supplemental display device 502 maydetermine the triggering phrase for the determined natural languageprocessing service (e.g., “Hey Siri” for the Ski service, “OK Google”for the Google Now service, “Hey Cortana” for the Cortana service,etc.).

The supplemental display device 502 may continuously monitor, using itsmicrophone, user voice inputs and in response to detecting thecorresponding trigger phrase from the user, transmit the trigger phrase(and a subsequent user voice command/request) and/or other triggerindication to the communication and synchronization application on thefirst computing device 504. The communication and synchronizationapplication on the first computing device 504 may then provide thetrigger phrase (and a subsequent user voice command/request) and/orother trigger indication to the natural language processing service. Theresponse from the natural language processing service may then betransmitted to the supplemental display device 502 for reproduction(e.g., a text, image, and/or video response may be displayed; an audioresponse may be played by the supplemental display device 502).

If a non-voice trigger of the natural language processing service isreceived via another input method on the supplemental display device 502(e.g., pressing of a home control for more than a threshold period oftime), optionally the trigger indication may be similarly transmitted tothe communication and synchronization application on the first computingdevice 504.

A user setting may be provided instructing the supplemental displaydevice 502 not to relay natural language processing service triggers tothe first computing device 504 and/or not to monitor the microphone onthe supplemental display device 502 in the absence of a manual userinstruction to do so.

Certain example aspects will now be discussed with reference to thefigures.

FIG. 6 illustrates an example communication and synchronizationarchitecture for the first computing device 504 and for the supplementaldisplay device 502. The first computing device 504 may include areplicator module and API, a haptic service (configured to detect usertouches and the locations thereof), a wireless headphone service, anunlocking service, an orientation service, an mDNS service, and acommunication channel management service. The supplemental displaydevice 502 may include a replicator module and API, a haptic service, asound service, an orientation service, an mDNS service, a powermanagement service, and a communication channel management service. Thevarious services and modules are described in greater detail elsewhereherein.

FIG. 7 illustrates an example device registration process which may beutilized the first time a first device (e.g., a user phone, Internet ofThings (IoT) device (e.g., a smart watch), small tablet computer,vehicle display, or other device) is “paired” with a supplementaldisplay device. The process may be initiated in response to detecting adevice pairing command. For example, a user may activate a pairingcommand on the supplemental display device, which may be detected by thesupplemental display device operating system. At block 702, thesupplemental display device launches an mDNS service or other resolvingservice. At block 704, the supplemental display device may utilize themDNS service or other resolving service to search for and identify thefirst device. At block 705, the supplemental display device may transmit(e.g., using a Bluetooth communication channel) an unlock command to thefirst device. A communication and synchronization application (“CSapplication”) on the first device may receive and detect the command andunlock the first device to so that the supplemental display device mayaccess and utilize the first device's software and hardware.

For example, the CS application on the first device may start aBluetooth-server-socket in a separate thread and wait for a connectionwith the supplemental display device. The supplemental display deviceconnects to the Bluetooth socket, and the supplemental display devicetransmits an unlock command to the first device. The first devicereceives the unlock command at the communication and synchronizationapplication, which in response, optionally turns on the first devicedisplay (if the display was off), and attempts to unlock the firstdevice.

If the user's has set security mechanisms in place for unlocking thefirst device (e.g., a PIN, a graphical unlock pattern, biometrics,etc.), the unlock input may optionally be performed via the supplementaldisplay device. For example, if the unlock input is to be provided viaan unlock user interface on the first device, the unlock user interfacemay be displayed on the supplemental display device. The user can thenprovide (e.g., via touch) the needed unlock input via the unlock userinterface on the supplemental display device. The supplemental displaydevice will transmit the unlock inputs to the CS application on thefirst device, which will use the unlock inputs to unlock the firstdevice (without the user having to touch or otherwise manually accessthe first device). If a biometric input is needed to unlock the firstdevice, the supplemental display device may prompt the user to providethe biometric input via a corresponding sensor on the supplementaldisplay device (e.g., a fingerprint sensor, a camera, iris reader, amicrophone, etc.). Once the supplemental display device receives thebiometric input, the supplemental display device will transmit thebiometric input to the CS application on the first device, which willuse the biometric input to unlock the first device. Optionally instead,the user may manually unlock the first device via user inputs on thefirst device itself.

At block 706, the supplemental display device may utilize a keygeneration module to generate two keys (e.g., a private key and a publickey). At block 708, the supplemental display device transmits one of thekeys (e.g., the public key) to the first device. At block 710, thesupplemental display device registers the first device (e.g., stores thekeys, receives and stores the telephone number of the first device, theMedia Access Control (MAC) address of the first device (which uniquelyidentifies the first device), the IMEI of the first device, the ICCID ofthe first device, the MEID of the first device, and/or otherinformation). The keys and/or the received MAC address of the firstdevice may be used in the future to enable the supplemental displaydevice to uniquely identify and authenticate the first device withouthaving to go through the registration process, and so as to establish asecure and safe wireless connection. For example, a MAC address receivedduring a registration operation may be compared with a MAC addressreceived during a later connection operation, and in response to adetermination that the MAC address received during the registrationoperation matches the MAC address received during the connectionoperation, verify the identity of the first device.

FIG. 8 illustrates an example replication process. At block 802, thesupplemental display device launches an mDNS service or other resolvingservice. At block 804, the supplemental display device identifies thefirst device. At block 806, the first device transmits its MAC addressto the supplemental display device. The first device may also transmitthe public key it received during the initial registration process backto the supplemental display device. Optionally, a key exchange isperformed where the supplemental display device and first deviceexchange their respective keys (that had been generated during theinitial registration process). At block 808, the supplemental displaydevice verifies the identity of the first device. At block 809, adetermination is made as to whether the verification is successful. Ifthe verification fails, at block 824, an error message may be generatedand presented to the user via the supplemental display device, and atblock 826 the replication process may terminate.

If the verification is successful, at block 810, an unlock service isexecuted at the supplemental display device, which commands the firstdevice (e.g., via Bluetooth) to unlock as similarly discussed elsewhereherein. At block 812, the supplemental display device may transmit arequest to the CS application on the first device for first deviceconfiguration information. For example, the first device configurationinformation may include some or all of the following: manufacturer,model, manufacturing data, release, operating system type (e.g., iOS,Windows, Android), operating system version. The first device maytransmit the requested configuration information to the supplementaldisplay device, which, at block 814, may receive and store in memory thereceived configuration information.

At block 816, the CS application on the first device may determine thecommunication environment of the first device. The determinedcommunication environment may be utilized to dynamically determine howto manage wireless communications. For example, the CS application maydetermine whether the first device is connected to the Internet viaWi-Fi (e.g., via a Wi-Fi access point), via cellular service (e.g.,LTE), or whether the first device is not connected to the Internet atall. The CS application on the first device may then transmit thecommunication environment data to the supplemental display device, whichreceives and stores the communication environment data.

As discussed below in greater detail with respect to FIG. 10, using thecommunication environment data, the supplemental display device maysetup multiple channels of communication. For example, the supplementaldisplay device may determine that certain commands or data are to betransmitted to the cloud server system discussed above, and certaincommands or data are to be transmitted to directly to the first device.By way of further example, the display device may determine that certaincommands or data are to be transmitted directly to the first device viaWi-Fi Direct, and certain other commands or data are to be transmittedusing Bluetooth, and still other commands or data are to be communicatedto an access point which may then communicate the commands to the firstdevice or the remote cloud server system. Optionally, the user may stillbe able to access services of the first device in a conventionalfashion.

For example, if the first device is a phone, the user may still pick upthe phone, dial a phone number, and place a call without interferingwith the supplemental display device's ability to access applications onthe first device or the ability to access content being accessed by suchapplications (e.g., video content, still image content, graphic content,text content, Web pages, etc.). This architecture enables thesupplemental display device to have the first device's applicationsexecuted in real-time.

At block 818, a haptic service is executed. In the context of utilizinga mouse HID pathway, the haptic service may optionally utilize absolutepositioning with respect to user touch inputs to avoid the technicalproblems of relative positioning, where the user would have to indicatea 0,0 position at the beginning of each session. At block 820,orientation synchronization between the supplemental display device andthe first device may be inhibited. For example, the supplemental displaydevice may transmit a command to the CS application not to communicateorientation information of the first device to the supplemental displaydevice. This enables images on the supplemental display device to beoriented based on the detected user orientation of the supplementaldisplay device being held by the user rather than the orientation of thefirst device. Optionally, a control may be provided via the supplementaldisplay device via which the user can instruct the supplemental displaydevice to synchronize its display orientation with that of the firstdevice.

At block 822, a replicator service may be executed on the first device.The replicator service may negotiate connection settings and may copywhat is displayed on the first device display to the supplementaldisplay device (e.g., utilizing an image capture application, such asMiracast or the Media Projection API) for display. A token may beprovided to the replicator service granting the ability to capturescreen content and/or audio content. At block 818, content from thefirst device is streamed to the supplemental display device (e.g.,utilizing a transport layer, such as User Datagram Protocol (UDP)). Atblock 826, an audio service is executed and audio content is streamedfrom the first device to the supplemental display device (e.g., usingBluetooth). At block 828, the video is streamed from the first device tothe supplemental display device (e.g., using Wi-Fi Direct).

The haptic service described above may be implemented differently fordifferent first device operating systems. For example, if the firstdevice utilizes the Apple® iOS operating system, all the available touchevents may be handled (e.g., tap, swipe, expand, pinch, rotate, etc.).For a given touch event, iOS may generate the following examplenotifications, as applicable: Begin, Move, End, etc. Using touch eventcoordinates generated on the supplemental display device side(considering the aspect ratio), touch events on the supplemental displaydevice may be applied on the iOS device. The touch event coordinates maybe transmitted (with comments that indicate the type of touch event,such as tap, expand, pinch, rotate, etc.) from the supplemental displaydevice to the iOS device. The iOS device detects receipt of suchcomments, and using the comments, determines what command (hapticfunction) to perform and with what parameters (e.g., positioning) toperform such command.

In order to ensure that the foregoing process is reliable andrepeatable, certain optional techniques may be utilized. The displays ofthe supplemental display device and the iOS device may be mapped so thatthey match. In particular, coefficients for the aspect ratio arecalculated. For example, a width coefficient may be calculated thatequals the supplemental display device screen width divided by the iOSdevice screen width. Similarly, a height coefficient may be calculatedthat equals the supplemental display device screen height divided by theiOS device screen height.

By way of illustrative example, if the iOS device has a 320×480 pixelscreen size and the supplemental display device has a 2560×1600 screensize, then:

-   -   Width coefficient=1600/320=5.0    -   Height coefficient=2560/480=5.3333

So, for example, if the user taps the middle of the supplemental displaydevice screen (where the screen is 2560×1600):

-   -   The tap screen coordinate is (2560/2, 1600/2)=(1280, 800)

Then, using the calculated width and height coefficients, the scaledcoordinate for iOS device may be calculated as follows:

-   -   Scaled coordinate=(1280/5, 800/5.33333)=(256, 150), which is the        middle of 320×480 display of the iOS device in this example.

Thus, the coefficients may be used to accurately map a touch on thedisplay of the supplemental display device to the corresponding positionon the iOS device, and the CS application on the iOS device mayvirtually “tap” the corresponding position on the iOS device to carryout the operation desired by the user. In addition, the foregoing datamay be packed into an object which is transmitted to the first device,further reducing latency issues.

Where the first device the Android operating system, the supplementaldisplay device may emulate a Bluetooth Human Input Device (HID), such asa track pad or mouse. A determination is made (e.g. by the supplementaldisplay device or the CS application on the first device) as to whattype of HID device is supported by the Android device, the determinationmay be stored for later use, and the supplemental display device may beconfigured to mimic the supported HID device using a HID descriptor.

If the Android device supports multiple HID devices, a determination maybe made if it supports a track pad, and if so, the supplemental displaydevice may be configured to mimic the track pad. If the Android devicedoes not support a track pad, a determination may be made if it supportsa mouse, and if so, the supplemental display device may be configured tomimic the mouse, and so on.

After the initial pairing, the supplemental display device may accessfrom memory an indication as to what type of HID device it shouldemulate without having to make the determination again.

Optionally, a HID descriptor for a touchpad may be embedded into thesupplemental display device firmware so that other devices (e.g.,Android devices) can find the ‘virtual HID’ and utilize it.

However, for an Android device, HID speed is an issue, which,conventionally, would make the user of a virtual mouse unusable in areal-world case. In other words, as the user moves the “mouse” (e.g.,the user's finger or a stylus) on the supplemental display devicedisplay, the movement/touch data transmitted to the first device is soerratic that that it will not adequately reflect the “mouse” movements.Conventionally, even if the user is moving the “mouse” a steady speed,the movement data appears as bursts of fast movements with stops in anon-uniform function. This technical problem is overcome byrecalculating into absolute coordinates. When the user taps thesupplemental display device screen, the 0,0 coordinate is set to theupper left hand corner of the supplemental display device screen and ofthe first device screen, and this enables the remapping of the grid.Further, the movement and comment data may be transmitted together as anobject to the first device.

FIG. 9 illustrates an example process of utilizing a service, such as aWeb service, to locate and stream content when the CS application isunable to ascertain the content location (e.g., the content URL). Atblock 902, a first device identifies a locator (e.g., a URL) related tovideo content being requested by the supplemental display device (e.g.,in response to a user request). For example, the request for videocontent may be initiated in response to a user input received via thetouchscreen or microphone of the supplemental display device. If thelocator precisely identifies the video content, then at block 904, thecontent is accessed. For example, if the an access point is availablevia which the supplemental display device can access the Internet, thesupplemental display device may access the content at the locator viathe access point, without having to route the request to or otherwiseutilize the first device. If an access point is not available, but thefirst computing device has Internet access via its cellular radio, thenthe first may access the video content at the locator, receives thestreamed video content, and wireles sly transmit the streamed videocontent to the supplemental display device via the CS application (e.g.,via Wi-Fi Direct).

If the locator does not precisely identify the video content (e.g., thelocator is a webpage URL on which video content is embedded), than atblock 906, the locator is transmitted to a remote system, such as thecloud-based server system discussed elsewhere herein. At block 908, theremote system receives the locator, accesses the document (e.g.,webpage) and/or website the locator is pointing to, parses the documentand/or website and identifies the precise locator (e.g., URL) for thevideo content. At block 910, the video content locator is returned bythe remote system to the first device.

At block 911, a determination is made as to whether the first device isconnected to a WIFI access point. If the first device is not connectedto a WIFI access point, then at block 912, the first device utilizes thelocator to access the video content, using its cellular radio, viastreaming, and, at block 914, transmits the raw streamed video content(optionally without decoding the video content) to the supplementaldisplay device via the CS application (e.g., via a Wi-Fi Directconnection). The supplemental display device may render and display thestreamed video content. By utilizing the raw video content streamed fromthe first device, rather than processed video content from the firstdevice, and performing any needed processing on the supplemental displaydevice, the playback quality may be higher, artifacts may be reduced oravoided altogether, and the user experience may be enhanced. However,optionally, the video content may be first processed on the first deviceand then transmitted to the supplemental display device for display.

If the first device is connected to a WIFI access point, then at block916, the supplemental display device access the WIFI access pointaddress from the first device. At block 918, the supplemental displaydevice utilizes the locator to access, via the access point, the videocontent via streaming, and processes, renders and displays the streamedvideo content, without having the video content routed via the firstdevice.

Although the foregoing example is directed to video content, optionallythe process may be similarly used for other content, such as audiocontent.

FIG. 10 illustrates an example process for utilizing informationregarding a communication environment for a first device in determininghow content is to be accessed and how communications are to be managed.At block 1002, the first device's communication environment isdetermined. For example, the CS application on the first device maydetermine and transmit to the supplemental display device thecommunication environment data (e.g., via a Bluetooth connection). Atblock 1004, a determination is made as to whether the first device canaccess the Internet via a Wi-Fi access point. If the first device cannotaccess the Internet via a Wi-Fi access point, then at block 1006, adetermination is made as to whether the first device can access acellular service (e.g., LTE).

If the first device cannot access either cellular service or a Wi-FiInternet access point, then at block 1014, then content already storedin memory of the first device's display may be transmitted (e.g.,streamed) for reproduction on the supplemental display device. Audiocontent may be transferred from the first device and to the supplementaldisplay device via a Bluetooth connection and video or GUI content maybe transferred from the first device to the supplemental display devicevia a Bluetooth connection or via a zero hop (e.g., Wi-Fi Direct)connection.

If the first device can access a Wi-Fi Internet access point, then atblock 1006, the supplemental display device may access (e.g., via aBluetooth connection) the locator (e.g., the URL) of the access pointthe first device is connected to, and the supplemental display devicemay connect to the same URL point. At block 1008, one or moreapplications are launched from the Web to the supplemental displaydevice, and respective Web services are accessed by the supplementaldisplay device via the Wi-Fi Internet access point. Advantageously, aWeb version of a given application may provide an enhanced userinterface and/or additional functionality as compared to the applicationinstalled on the first device. The supplemental display device mayreceive login information (e.g., a UserID and/or password) for the useraccount associated with the given application and may submit the logininformation via a user interface or API of the Web version of theapplication to login and enable the user to access the user's account,page, and/or other information. Video may be streamed directly from theaccess point to the supplemental display device. Optionally, audiocontent may be transferred from the first device and to the supplementaldisplay device via the already existing Bluetooth connection. Whenexiting out of an app back to the grid of app icons, the grid userinterface of apps generated by the supplemental display device may bepresented, solving the problem of traditional mirroring andcompatibility issues of aspect ratio, scale of icons/text, lost areas ofscreen, cropping, etc.

If it is determined that the first device has access to a cellularservice but not a Wi-Fi access point, then at block 1016, the firstdevice connects to the supplemental display device using a zero hopconnection (e.g., via a Wi-Fi Direct), in addition to the already-openedBluetooth connection. Audio content may be transferred from the firstdevice and to the supplemental display device via the zero hopconnection (e.g., Wi-Fi Direct). At block 1018, one or more applicationsare launched and access, using the first device's cellular radio,respective Web services via the cellular service. At block 1020, dataaccessed by app(s) on the first device are transmitted to thesupplemental display device via the zero hop connection. The userinterfaces of launched applications may also be transmitted from thefirst device to the supplemental display device for display.

Thus, for example, a determination may be made as to the first device'scommunication environment, and then, if available, utilize the accesspoint option to access the Internet, and if the access point option isnot available, utilize the first device's cellular service (e.g., LTE)to access the Internet, and if the access point and cellular service areboth not available, utilizing content mirroring without accessing theInternet. The process may constantly monitor the communicationenvironment, so that if there is a change in the availability of accesspoint or cellular service availability, the process may repeat and thecommunications be according redirected in real time and seamlessly sothat there is no user-perceived change in their use.

An example process for configuring the layout of app icons (or text)will be described. A first virtual grid is generated for the firstdevice display and a second virtual grid is generated for thesupplemental display device display. The vertical and horizontalplacement of the grid lines for the first and second grids are spaced sothat they define grid squares or rectangles of the same size in thefirst and second grids. However, if the supplemental display device hasa larger display than the first device, then the supplemental displaydevice grid will have more grid lines and a greater number of grid areas(e.g., squares or rectangles) defined by the grid lines, even thougheach grid area is the same size as that of the first grid for the firstdevice.

For example, if the display of the supplemental display device has fourtimes the area of the display of the first device, then the second gridmay have four times as many grid lines as the first grid. Applicationicons from the first device are accessed by the supplemental displaydevice and displayed by the supplemental display device using the secondgrid (optionally using the same layout as the first device's layout, asgenerated by the CS application), with the icons being the same size orsubstantially the same size (e.g., +/−5% or +/−10%) as on the firstdevice. This makes additional display space available on thesupplemental display device (e.g., for work/entertainment or to displayadditional icons at the same time).

By contrast, conventional mirroring may scale the icons by a factorproportional to the increase in display size. Thus, if a source deviceis being mirrored on a sink device that has a display that is four timeslarger than that of the source device, the icons displayed by the sinkdevice may be four times the size as that of the icons displayed on thesource device. This can result in icons that are awkward in size andpixilation, and does not make additional display space available on thesink device display (e.g., for work/entertainment or to displayadditional icons at the same time).

FIG. 11 illustrates an example process of determining characteristics ofthe first device so that the supplemental display device can adequatelycommunicate with and control the first device. At block 1100, thesupplemental display device queries the CS application on the firstdevice to determine characteristics of the first device and receivescorresponding information from the CS application. At block 1102, thesupplemental display device utilizes the received information todetermine the device type (e.g., manufacturer, model, release, height ofdisplay in pixels, width of display in pixels, etc.). At block 1104, thesupplemental display device utilizes the information to determine thedevice operating system (e.g., iOS or Android). At block 1106, thesupplemental display device utilizes the information to identify one ormore of the HIDs supported by the device (e.g., track pad, mouse, etc.).At block 1108, the supplemental display device utilizes the device typeinformation, the operating system type, and the supported HIDs tocalculate aspect ratio coefficients, to determine how to reduce powerconsumption of the first device (e.g., whether to hibernate the firstdisplay device or reduce the first display device brightness to zero oras low as permitted), how to grab audio content on the first device, howto grab display content on the first device, to determine what HID toemulate, and/or to perform other functions described herein.

Thus, systems and methods are provided to enable a supplemental displaydevice to reliably and accurately display data from a paired device(e.g., via a larger display), control the paired device, managecommunications with the paired device, and/or provide other functionsdescribed herein.

The methods and processes described herein may have fewer or additionalsteps or states and the steps or states may be performed in a differentorder. Not all steps or states need to be reached. The methods andprocesses described herein may be embodied in, and fully or partiallyautomated via, software code modules executed by one or more generalpurpose computers. The code modules may be stored in any type ofcomputer-readable medium or other computer storage device. Some or allof the methods may alternatively be embodied in whole or in part inspecialized computer hardware. The systems described herein mayoptionally include displays, user input devices (e.g., touchscreen,keyboard, mouse, voice recognition, etc.), network interfaces, etc.

The results of the disclosed methods may be stored in any type ofcomputer data repository, such as relational databases and flat filesystems that use volatile and/or non-volatile memory (e.g., magneticdisk storage, optical storage, EEPROM and/or solid state RAM).

The various illustrative logical blocks, modules, routines, andalgorithm steps described in connection with the embodiments disclosedherein can be implemented as electronic hardware, computer software, orcombinations of both. To clearly illustrate this interchangeability ofhardware and software, various illustrative components, blocks, modules,and steps have been described above generally in terms of theirfunctionality. Whether such functionality is implemented as hardware orsoftware depends upon the particular application and design constraintsimposed on the overall system. The described functionality can beimplemented in varying ways for each particular application, but suchimplementation decisions should not be interpreted as causing adeparture from the scope of the disclosure.

Moreover, the various illustrative logical blocks and modules describedin connection with the embodiments disclosed herein can be implementedor performed by a machine, such as a processor device, a digital signalprocessor (DSP), an application specific integrated circuit (ASIC), afield programmable gate array (FPGA) or other programmable logic device,discrete gate or transistor logic, discrete hardware components, or anycombination thereof designed to perform the functions described herein.A processor device can be a microprocessor, but in the alternative, theprocessor device can be a controller, microcontroller, or state machine,combinations of the same, or the like. A processor device can includeelectrical circuitry configured to process computer-executableinstructions. In another embodiment, a processor device includes an FPGAor other programmable device that performs logic operations withoutprocessing computer-executable instructions. A processor device can alsobe implemented as a combination of computing devices, e.g., acombination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. Although described hereinprimarily with respect to digital technology, a processor device mayalso include primarily analog components. For example, some or all ofthe rendering techniques described herein may be implemented in analogcircuitry or mixed analog and digital circuitry. A computing environmentcan include any type of computer system, including, but not limited to,a computer system based on a microprocessor, a mainframe computer, adigital signal processor, a portable computing device, a devicecontroller, or a computational engine within an appliance, to name afew.

The elements of a method, process, routine, or algorithm described inconnection with the embodiments disclosed herein can be embodieddirectly in hardware, in a software module executed by a processordevice, or in a combination of the two. A software module can reside inRAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory,registers, hard disk, a removable disk, a CD-ROM, or any other form of anon-transitory computer-readable storage medium. An exemplary storagemedium can be coupled to the processor device such that the processordevice can read information from, and write information to, the storagemedium. In the alternative, the storage medium can be integer to theprocessor device. The processor device and the storage medium can residein an ASIC. The ASIC can reside in a user terminal. In the alternative,the processor device and the storage medium can reside as discretecomponents in a user terminal.

Conditional language used herein, such as, among others, “can,” “may,”“might,” “may,” “e.g.,” and the like, unless specifically statedotherwise, or otherwise understood within the context as used, isgenerally intended to convey that certain embodiments include, whileother embodiments do not include, certain features, elements and/orsteps. Thus, such conditional language is not generally intended toimply that features, elements and/or steps are in any way required forone or more embodiments or that one or more embodiments necessarilyinclude logic for deciding, with or without other input or prompting,whether these features, elements and/or steps are included or are to beperformed in any particular embodiment. The terms “comprising,”“including,” “having,” and the like are synonymous and are usedinclusively, in an open-ended fashion, and do not exclude additionalelements, features, acts, operations, and so forth. Also, the term “or”is used in its inclusive sense (and not in its exclusive sense) so thatwhen used, for example, to connect a list of elements, the term “or”means one, some, or all of the elements in the list.

Disjunctive language such as the phrase “at least one of X, Y, Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y, or Z, or any combination thereof (e.g., X, Y, and/or Z).Thus, such disjunctive language is not generally intended to, and shouldnot, imply that certain embodiments require at least one of X, at leastone of Y, or at least one of Z to each be present.

While the phrase “click” may be used with respect to a user selecting acontrol, menu selection, or the like, other user inputs may be used,such as voice commands, text entry, gestures, etc. User inputs may, byway of example, be provided via an interface or in response to a prompt(e.g., a voice or text prompt). By way of example an interface mayinclude text fields, wherein a user provides input by entering text intothe field. By way of further example, a user input may be received via amenu selection (e.g., a drop down menu, a list or other arrangement viawhich the user can check via a check box or otherwise make a selectionor selections, a group of individually selectable icons, a menuselection made via an interactive voice response system, etc.). When theuser provides an input or activates a control, a corresponding computingsystem may perform a corresponding operation (e.g., store the userinput, process the user input, provide a response to the user input,etc.). Some or all of the data, inputs and instructions provided by auser may optionally be stored in a system data store (e.g., a database),from which the system may access and retrieve such data, inputs, andinstructions. The notifications and user interfaces described herein maybe provided via a Web page, a dedicated or non-dedicated phoneapplication, computer application, a short messaging service message(e.g., SMS, MMS, etc.), instant messaging, email, push notification,audibly, and/or otherwise.

The user terminals described herein may be in the form of a mobilecommunication device (e.g., a cell phone, a VoIP equipped mobile device,etc.), laptop, tablet computer, interactive television, game console,media streaming device, head-wearable display, virtual realitydisplay/headset, augmented reality display/headset, networked watch,etc. The user terminals may optionally include displays, user inputdevices (e.g., touchscreen, keyboard, mouse, voice recognition, etc.),network interfaces, etc.

Certain aspects are described in U.S. patent application Serial No.______, entitled “SYSTEMS AND METHODS FOR A SUPPLEMENTAL DISPLAYSCREEN,” filed ______, and corresponding to Attorney Docket No.BRMOR.003P1, which is incorporated by reference herein in its entirety.While the above detailed description has shown, described, and pointedout novel features as applied to various embodiments, it can beunderstood that various omissions, substitutions, and changes in theform and details of the devices or algorithms illustrated can be madewithout departing from the spirit of the disclosure. As can berecognized, certain embodiments described herein can be embodied withina form that does not provide all of the features and benefits set forthherein, as some features can be used or practiced separately fromothers.

1. A computer-implemented method comprising: determining by a firstdevice configuration data for the first device, wherein the first devicehosts a communication and synchronization application; wirelesslytransmitting the determined first device configuration data from thefirst device to a supplementary display device; determining, by thefirst device, a communication environment of the first device andgenerating corresponding communication environment data, thecommunication environment data indicating: whether the first device iswirelessly connected to a WIFI access point, and whether the firstdevice has access to a cellular network; based at least in part on adetermination that the first device is wirelessly connected to a WIFIaccess point: enabling the supplementary display device to connectdirectly to the WIFI access point by wirelessly transmitting datarelated to the WIFI access point to the supplementary display device;based at least in part on a determination that the first device hasaccess to the cellular network and is not wirelessly connected to a WIFIaccess point: enabling the supplementary display device to accessnetwork resources using the first device's access to the cellularnetwork; receiving from the supplementary display device by the firstdevice touch data corresponding to a user touch input on a touchscreenof the supplementary display device, the touch data including at leastcoordinate data corresponding to the user touch input; determining bythe first device an action to be taken in response to the touch datareceived from the supplementary display device; taking, by the firstdevice, the determined action; and wirelessly transmitting contentassociated with the taken action from the first device to thesupplementary display device.
 2. The method as defined in claim 1, themethod further comprising: receiving from the supplementary displaydevice, via a first wireless interface, at the first device an unlockinstruction; and unlocking the first device at least partly in responseto the wirelessly received unlock instruction.
 3. The method as definedin claim 1, wherein transmitting content associated with the takenaction from the first device to the supplementary display device furthercomprises: transmitting video data from the first device to thesupplementary display device using a WIFI connection; and transmittingaudio data, associated with the video data, from the first device to thesupplementary display device using a Bluetooth connection.
 4. Acomputer-implemented method comprising: determining by a first deviceconfiguration data for the first device, wherein the first device hostsa communication and synchronization application; wirelessly transmittingthe determined first device configuration data from the first device toa supplementary display device; determining, by the first device, acommunication environment of the first device and generatingcorresponding communication environment data, the communicationenvironment data indicating whether the first device is wirelesslyconnected to a WIFI access point and whether the first device has accessto a cellular network; wirelessly transmitting the first devicecommunication environment data from the first device to thesupplementary display device, the communication environment dataindicating whether the first device is wirelessly connected to a WIFIaccess point and whether the first device has access to a cellularnetwork; receiving from the supplementary display device by the firstdevice touch data corresponding to a user touch input on a touchscreenof the supplementary display device, the touch data including at leastcoordinate data corresponding to the user touch input; determining bythe first device an action to be taken in response to the touch datareceived from the supplementary display device; taking, by the firstdevice, the determined action; wirelessly transmitting contentassociated with the taken action from the first device to thesupplementary display device; determining by the first device, fromsecond touch data received from the supplementary display device, that arequest for a video embedded in a webpage is being requested; attemptingto identify, by the first device, a locator for the requested video; atleast partly in response to failing to identify the locator for therequested video, transmitting from the first device a request to aremote server system to identify the locator for the requested video;receiving the locator for the requested video from the remote serversystem; utilizing the received locator for the requested video to enablethe supplementary display device to access the requested video.
 5. Themethod as defined in claim 1, wherein the taken action comprisesaccessing, by the first device, video content over a network, the methodfurther comprising streaming the accessed video content from the firstdevice to the supplementary display device without changing a scaling orresolution of the accessed video content.
 6. The method as defined inclaim 1, wherein determining by the first device an action to be takenin response to the touch data further comprises determining if thecoordinate data corresponds to an application, and wherein taking, bythe first device, the determined action, further comprises launching theapplication.
 7. The method as defined in claim 1, wherein determining,by the first device, an action to be taken in response to the touch datacorresponding to the user touch input on the touchscreen of thesupplementary display device further comprises determining if the userperformed a swipe, expand, pinch, or rotate gesture on the touchscreenof the supplementary display device.
 8. A computer system, comprising: atouchscreen display; computing hardware; a first wireless networkinginterface; a second wireless networking interface; a third wirelessnetworking interface; non-transitory memory that stores instructionsthat when executed by the computing hardware cause the computer systemto perform operations comprising: accessing configuration data of thecomputer system; wirelessly transmitting the configuration data to asupplementary display device using the first wireless networkinginterface or the second wireless networking interface; determining acommunication environment of the computer system, comprisingdetermining: whether the computer system is wirelessly connected to anetwork access point using the second wireless networking interface, andindicating whether the computer system has access to a cellular networkvia the third wireless networking interface; based at least in part on adetermination that the computer system is wirelessly connected to thenetwork access point: enabling the supplementary display device toconnect directly to the network access point by wirelessly transmittingdata related to the network access point to the supplementary displaydevice; based at least in part on a determination that the first devicehas access to the cellular network and is not wirelessly connected to anetwork access point: enabling the supplementary display device toaccess network resources using the computer system's access to thecellular network; receiving from the supplementary display device touchdata corresponding to a user touch input on a touchscreen of thesupplementary display device, the touch data including at least touchcoordinate data; determining at the computer system an action to betaken in response to the touch data, including at least the touchcoordinate data received from the supplementary display device; andtaking, by the computer system, the determined action.
 9. The system asdefined in claim 8, the operations further comprising: receiving fromthe supplementary display device, via the first wireless networkinginterface or the second wireless networking interface, an unlockinstruction; and performing an unlocking operation at least partly inresponse to the received unlock instruction.
 10. The system as definedin claim 8, the operations further comprising: transmitting video datato the supplementary display device using the second wireless networkinginterface; and transmitting audio data, associated with the video data,to the supplementary display device using the first wireless networkinginterface.
 11. A computer system, comprising: a touchscreen display;computing hardware; a first wireless networking interface; a secondwireless networking interface; a third wireless networking interface;non-transitory memory that stores instructions that when executed by thecomputing hardware cause the computer system to perform operationscomprising: accessing configuration data of the computer system;wirelessly transmitting the configuration data to a supplementarydisplay device using the first wireless networking interface or thesecond wireless networking interface; determining a communicationenvironment of the computer system and generating correspondingcommunication environment data, the communication environment dataindicating whether the computer system is wirelessly connected to anetwork access point using the second wireless networking interface andindicating whether the computer system has access to a cellular networkvia the third wireless networking interface; wirelessly transmitting thecommunication environment data to the supplementary display device;receiving from the supplementary display device touch data correspondingto a user touch input on a touchscreen of the supplementary displaydevice, the touch data including at least touch coordinate data;determining at the computer system an action to be taken in response tothe touch data, including at least the touch coordinate data receivedfrom the supplementary display device; taking, by the computer system,the determined action; determining, from second touch data received fromthe supplementary display device, that a request for a video embedded ina webpage is being requested; attempting to identify a locator in thewebpage for the requested video; at least partly in response to failingto identify the locator for the requested video, transmitting a requestto a remote system to identify the locator for the requested video;receiving the locator for the requested video from the remote system;and utilizing the received locator for the requested video to enable thesupplementary display device to access the requested video.
 12. Thesystem as defined in claim 8, wherein the taken action comprisesaccessing video content over a network, the operations furthercomprising streaming the accessed video content to the supplementarydisplay device without changing a scaling or resolution of the accessedvideo content.
 13. The system as defined in claim 8, wherein determiningan action to be taken in response to the touch data corresponding to theuser touch input on the touchscreen of the supplementary display devicefurther comprises determining if the coordinate data corresponds to anapplication, and wherein taking the determined action, further compriseslaunching the application.
 14. The system as defined in claim 8, whereindetermining an action to be taken in response to the touch datacorresponding to the user touch input on the touchscreen of thesupplementary display device further comprises determining if the userperformed a swipe, expand, pinch, or rotate gesture on the touchscreenof the supplementary display device.
 15. A non-transitory memory thatstores instructions that when executed by a computing system cause thecomputer system to perform operations comprising: accessingconfiguration data of the computer system; wirelessly transmitting theconfiguration data to a supplementary display device using a firstwireless networking interface or a second wireless networking interface;determining a communication environment of the computer system,comprising determining: whether the computer system is wirelesslyconnected to a network access point using the second wireless networkinginterface, and indicating whether the computer system has access to acellular network via a third wireless networking interface; based atleast in part on a determination that the computer system is wirelesslyconnected to the network access point: enabling the supplementarydisplay device to connect directly to the network access point bywirelessly transmitting data related to the network access point to thesupplementary display device; based at least in part on a determinationthat the computer system has access to the cellular network and is notwirelessly connected to a network access point: enabling thesupplementary display device to access network resources using thecomputer system's access to the cellular network; receiving from thesupplementary display device touch data corresponding to a user touchinput on a touchscreen of the supplementary display device, the touchdata including at least touch coordinate data; determining an action tobe taken in response to the touch data, including at least the touchcoordinate data, received from the supplementary display device; andtaking the determined action.
 16. The non-transitory memory as definedin claim 15, the operations further comprising: transmitting an item ofcontent to a first plurality of supplementary display devices andcausing the item of content to be presented on the first plurality ofsupplementary display devices at the same time.
 17. The non-transitorymemory as defined in claim 15, the operations further comprising:transmitting video data to the supplementary display device using thesecond wireless networking interface; and transmitting audio data,associated with the video data, to the supplementary display deviceusing the first wireless networking interface.
 18. A non-transitorymemory that stores instructions that when executed by a computing systemcause the computer system to perform operations comprising: accessingconfiguration data of the computer system; wirelessly transmitting theconfiguration data to a supplementary display device using a firstwireless networking interface or a second wireless networking interface;determining a communication environment of the computer system andgenerating corresponding communication environment data, thecommunication environment data indicating whether the computer system iswirelessly connected to a network access point using the second wirelessnetworking interface and indicating whether the computer system hasaccess to a cellular network via a third wireless networking interface;wirelessly transmitting the communication environment data to thesupplementary display device; receiving from the supplementary displaydevice touch data corresponding to a user touch input on a touchscreenof the supplementary display device, the touch data including at leasttouch coordinate data; determining an action to be taken in response tothe touch data, including at least the touch coordinate data, receivedfrom the supplementary display device; taking the determined action;determining, from touch data received from the supplementary displaydevice, that a request for a video embedded in a webpage is beingrequested; attempting to identify a locator in the webpage for therequested video; at least partly in response to failing to identify thelocator for the requested video, transmitting a request to a remotesystem to identify the locator for the requested video; receiving thelocator for the requested video from the remote system; and utilizingthe received locator for the requested video to enable the supplementarydisplay device to access the requested video.
 19. The non-transitorymemory as defined in claim 15, wherein determining an action to be takenin response to the touch data corresponding to the user touch input onthe touchscreen of the supplementary display device further comprisesdetermining if the coordinate data corresponds to an application, andwherein taking the determined action further comprises launching theapplication.
 20. The non-transitory memory as defined in claim 15,wherein determining an action to be taken in response to the touch datacorresponding to the user touch input on the touchscreen of thesupplementary display device further comprises determining if the userperformed a swipe, expand, pinch, or rotate gesture on the touchscreenof the supplementary display device.
 21. A non-transitory memory thatstores instructions that when executed by a computing system cause thecomputer system to perform operations comprising: determining: whetherthe computer system is wirelessly connected to a network access pointusing a first wireless networking interface, and whether the computersystem has access to a cellular network via a second wireless networkinginterface; based at least in part on a determination that the computersystem is wirelessly connected to the network access point: enabling asupplementary display device to connect directly to the network accesspoint by wirelessly transmitting data related to the network accesspoint to the supplementary display device; based at least in part on adetermination that the computer system has access to the cellularnetwork and is not wirelessly connected to a network access point:enabling the supplementary resources using the computer system's accessto the cellular network; identifying one or more applications installedon the computing system; accessing icon images corresponding to the oneor more applications installed on the computing system; wirelesslytransmitting the icon images corresponding to the one or moreapplications installed on the computing system to a supplementarydisplay device; receiving from the supplementary display device touchdata corresponding to a user touch input on a touchscreen of thesupplementary display device, the touch data including at least touchcoordinate data; determining based at least in part on the touch data,including at least the coordinate data, whether an application on thecomputer system is to be launched; in response to determining that anapplication on the computer system is to be launched, launching theapplication; and transmitting a user interface corresponding to thelaunched application to the supplementary display device.
 22. Thenon-transitory memory as defined in claim 21, the operations furthercomprising: transmitting an item of content to a first plurality ofsupplementary display devices and causing the item of content to bepresented on the first plurality of supplementary display devices at thesame time.
 23. The non-transitory memory as defined in claim 21, theoperations further comprising: transmitting video data and associatedaudio data to the supplementary display device, wherein the audio datais transmitted to the supplementary display device using a differentnetwork interface than that used to transmit the video data to thesupplementary display device.
 24. A non-transitory memory that storesinstructions that when executed by a computing system cause the computersystem to perform operations comprising: identifying one or moreapplications installed on the computing system; accessing icon imagescorresponding to the one or more applications installed on the computingsystem; wirelessly transmitting the icon images corresponding to the oneor more applications installed on the computing system to asupplementary display device; receiving from the supplementary displaydevice touch data corresponding to a user touch input on a touchscreenof the supplementary display device, the touch data including at leasttouch coordinate data; determining based at least in part on the touchdata, including at least the coordinate data, whether an application onthe computer system is to be launched; in response to determining thatan application on the computer system is to be launched, launching theapplication; transmitting a user interface corresponding to the launchedapplication to the supplementary display device; determining, from touchdata received from the supplementary display device, that a request fora video embedded in a webpage is being requested; attempting to identifya locator in the webpage for the requested video; at least partly inresponse to failing to identify the locator for the requested video,transmitting a request to a remote system to identify the locator forthe requested video; receiving the locator for the requested video fromthe remote system; and utilizing the received locator for the requestedvideo to enable the supplementary display device to access the requestedvideo.
 25. The non-transitory memory as defined in claim 21, theoperations further comprising: wirelessly accessing video content from aremote source in response to the touch data from the supplementarydisplay device; and streaming the accessed video content to thesupplementary display device without changing a scaling or resolution ofthe accessed video content.
 26. The non-transitory memory as defined inclaim 21, the operations further comprising: determining, from secondtouch data received from the supplementary display device, if the userperformed a swipe, expand, pinch, or rotate gesture on the touchscreenof the supplementary display device, and taking a corresponding action.